Why Back propagation in RNN isn't effective

If you observe, to compute the gradient wrt the previous hidden state, which is the downstream gradient, the upstream gradient flows through the tanh non-linearity and gets multiplied by the weight matrix. Now, since this downstream gradient flows back across time steps, it means the computation happens over and over again at every time step. There are a couple of problems with this:

Since we’re multiplying over and over again by the weight matrix, the gradient will be scaled up or down depending on the largest singular value of the matrix: if the singular value is greater than 1, we’ll face an exploding gradient problem, and if it’s less than 1, we’ll face a vanishing gradient problem.

Now, the gradient passes through the tanh non-linearity which has saturating regions at the extremes. It means the gradient will essentially become zero if it has a high or low value once it passes through the non-linearity — so the gradient cannot propagate effectively across long sequences and it leads to ineffective optimization.

There is a way to avoid the exploding gradient problem by essentially “clipping” the gradient if it crosses a certain threshold. However, RNN still cannot be used effectively for long sequences.

References:

https://towardsdatascience.com/backpropagation-in-rnn-explained-bdf853b4e1c2#:~:text=You%20see%2C%20a%20RNN%20essentially,where%20they%20are%20summed%20up.

journalctl logging

Journalctl is a utility for querying and displaying logs from journald, systemd’s logging service. Since journald stores log data in a binary format instead of a plaintext format, journalctl is the standard way of reading log messages processed by journald.

In the following paragraphs, we’ll show you several ways of using journalctl to retrieve, format, and analyze your logs. These methods can be used on their own or in combination with other commands to refine your search. 

When run without any parameters, the following command will show all journal entries, which can be fairly long:

$ journalctl

The entries will start with a banner similar to this which shows the time span covered by the log.

-- Logs begin at Tue 2019-06-11 08:11:07 EDT, end at Mon 2019-06-24 15:18:11 EDT. --

Journalctl splits the results into pages, similar to the less command in Linux. You can navigate using the arrow keys, the Page Up/Page Down keys, and the space bar. To quit navigation, press the Q key.

Long entries are printed to the width of the screen and truncated off at the end if they don’t fit. The cut-off portion can be viewed using the left and right arrow keys.

Boot Messages

Journald tracks each log to a specific system boot. To limit the logs shown to the current boot, use the -b switch.

$ journalctl -b

You can view messages from an earlier boot by passing in its offset from the current boot. For example, the previous boot has an offset of -1, the boot before that is -2, and so on. Here, we are retrieving messages from the last boot:

$ journalctl -b -1

To list the boots of the system, use the following command.

$ journalctl --list-boots

It will show a tabular result like this.

-3 5a035370cc264015a5afcad6e310769f Sun 2019-06-23 09:27:30 EDT—Sun 2019-06-23 11:26:45 EDT

-2 ff65fc7baac14b429a4f41828db669d4 Sun 2019-06-23 11:59:55 EDT—Sun 2019-06-23 12:29:46 EDT

-1 2d54fbdb9fc04087930fd7543f57e922 Sun 2019-06-23 20:29:15 EDT—Sun 2019-06-23 23:01:43 EDT

0 aa2a1cf3cc2143b2a0245403739a336e Mon 2019-06-24 09:23:50 EDT—Mon 2019-06-24 15:18:11 EDT

The first field is the offset (0 being the latest boot, -1 being the boot before that, and so on), followed by a Boot ID (a long hexadecimal number), followed by the time stamps of the first and the last messages related to that boot.

Time Ranges

To see messages logged within a specific time window, we can use the --since and --until options. The following command shows journal messages logged within the last hour.

$ journalctl --since "1 hour ago"

To see messages logged in the last two days, the following command can be used.

$ journalctl --since "2 days ago"

The command below will show messages between two dates and times. All messages logged on or after the since parameter and logged on or before the until parameter will be shown.

$ journalctl --since "2015-06-26 23:15:00" --until "2015-06-26 23:20:00"

For greater accuracy, format the date and time as “YYYY-MM-DD HH:MM:SS”. You can also use any format that follows the systemd.time specification.

By Unit

To see messages logged by any systemd unit, use the -u switch. The command below will show all messages logged by the Nginx web server. You can use the --since and --until switches here to pinpoint web server errors occurring within a time window.

$ journalctl -u nginx.service

The -u switch can be used multiple times to specify more than one unit source. For example, if you want to see log entries for both nginx and mysql, the following command can be used.

$ journalctl -u nginx.service -u mysql.service

Follow or Tail

Journalctl can print log messages to the console as they are added, much like the Linux tail command. To do this, add the -f switch,

$ journalctl -f

For example, this command “follows” the mysql service log.

$ journalctl -u mysql.service -f

To stop following and return to the prompt, press Ctrl+C.

Like the tail command, the -n switch will print the specified number of most recent journal entries. In the command below, we are printing the last 50 messages logged within the last hour.

$ journalctl -n 50 --since "1 hour ago"

The -r parameter shows journal entries in reverse chronological order, so the latest messages are printed first. The command below shows the last 10 messages from the sshd daemon, listed in reverse order.

$ journalctl -u sshd.service -r -n 1

How to check serial number of Cisco Products

The link in references give a good detail of commands 

Essentially, show version and show inventory commands can give these details. 

PID is model name, SN is the serial number. Per interface, this will be available. show version normally show the chassis PID. 

references

https://blog.router-switch.com/2020/08/how-to-check-the-serial-number-of-cisco-products/

Static configuration of DHCP on the IOS-XE devices

it is quite simple, below sequence will do 

Prior to running the commands, needs to have the static-bindings file at the tftp server location with the mapping information like this below 

*time* Sep 27 2022 03:52 PM

*version* 2

!IP address    Type    Hardware address     Lease expiration

10.89.197.7 /21   1       d4ad.71c2.f990       Infinite

*end*

Now can run the below on the device.

enable

configure terminal

ip dhcp pool cemautomation

origin file tftp://<tftp server ip>/static-bindings

end

show ip dhcp binding

references:

https://www.cisco.com/en/US/docs/ios/12_4t/ip_addr/configuration/guide/htdhcpsv.html#wp1093993

Cisco Router how to view all interfaces

Type "show interfaces" and press the "Enter" key to view all MAC addresses on the router. Press the "Space Bar" to scroll through the output one page at a time. Press the "Enter" key to scroll through the output one line at a time.

references 

https://ciscorouterreview.weebly.com/how-to-find-all-ip--mac-addresses-on-a-router.html#:~:text=Type%20%22show%20interfaces%22%20and%20press,one%20line%20at%20a%20time.

PumpKin TFTP client

PumpKIN is an open source, fully functional, free TFTP server and TFTP client, which implements TFTP according to RFC1350 . It also implements block size option, which allows transfer of files over 32MB, as well as transfer size and transfer timeout options described in RFC2348 and RFC2349 .

the main PumpKIN features are:

Fully standard-compliant TFTP file transfer

Unlimited simultaneous transfers both for TFTP server and client

Support for TFTP blocksize option allows transfer of large files if both TFTP server and client support it

Convenient GUI

Combines TFTP server and TFTP client

Originally developed for Windows 95, it reportedly runs on all Win32 platforms: Windows 98, Windows NT, ME, XP, now also ported to Mac OS X (so far only tested on Mountain Lion)

Can run in background, taking up a 256 pixels of screen nicely packed as a 16x16 square in your notification tray area (windows only)

Open source for those willing to add missing features, fix bugs and examine code for potential flaws and fun

You’re free to torture it the way you want to as long as you preserve original author’s credentials

It would cost you nothing unless you’re willing to monetarily express your gratitude and make a donation (yes, it means “free” or “freeware”, just go and download it)

The download size is about that of the high quality screenshot below (windows only — Mac version is bigger, due to graphics supplied for way too many resolutions).

Note that PumpKIN is not an FTP server, neither it is an FTP client, it is a TFTP server and TFTP client. TFTP is not FTP, these are different protocols. TFTP, unlike FTP, is used primarily for transferring files to and from the network equipment (e.g. your router, switch, hub, whatnot firmware upgrade or backup, or configuration backup and restore) that supports using of TFTP server for, not for general purpose serving downloadable files or retrieving files from the FTP servers around the world.

references:

https://kin.klever.net/pumpkin/

DHCP sequence diagram , a great tutorial

DHCP is normally used to assign a computer its IP address, as well as other parameters such as the address of the local router. Your computer, the client, uses the DHCP protocol to communicate with a

DHCP server on the local network. Other computers on the local network also interact with the DHCP

server. In deployments, there are several variations. For example, the local agent may be a DHCP relay

that relays messages between local computers and a remote DHCP server. Or the DHCP server may be

replicated for reliability, so that there are two or more local DHCP servers. For our purposes, it is sufficient to think about a single DHCP server.

The complete DHCP exchange involves four types of packets: Discover, for your computer to locate the

DHCP server; Offer, for the server to offer an IP address; Request, for your computer to ask for an offered address; and Ack, for the server to grant the address lease. However, when a computer is re-establishing its IP address on a network that it has previously used, it may perform a short exchange involving only two types of DHCP packets: Request, to ask for the same IP address as from the same server

as was used before; and ACK for the server to grant the address lease.

references:

https://kevincurran.org/com320/labs/wireshark/lab-dhcp.pdf