Network Communication

Protocols
The rules you should care about
Protocols
Protocols
A communication protocol is a system of rules that allows two or more entities of a communications system to transmit
information via any kind of variation of a physical quantity. The protocol defines the rules, syntax, semantics and
synchronization of communication and possible error recovery methods.

Communicating systems use well-defined formats for exchanging various messages. Each message has an exact meaning
intended to elicit a response from a range of possible responses pre-determined for that particular situation. The specified
behavior is typically independent of how it is to be implemented. Communication protocols have to be agreed upon by the
parties involved
How data is sent
Circuit switching is a method of implementing a telecommunications network in which two network nodes establish a
dedicated communications channel (circuit) through the network before the nodes may communicate.
The circuit guarantees the full bandwidth of the channel and remains connected for the duration of the communication
session. The circuit functions as if the nodes were physically connected as with an electrical circuit. Circuit switching
originated in analog telephone networks where the network created a dedicated circuit between two telephones for the
duration of a telephone call.
How data is sent
Packet switching is a method of grouping data that is transmitted over a digital network into packets. Packets are made of
a header and a payload. Data in the header is used by networking hardware to direct the packet to its destination, where
the payload is extracted and used by application software. Packet switching is the primary basis for data communications in
computer networks worldwide.

Basically, that means that information is splitted into packets of predefined size and sent over network. In case there’s some
congestion on the way, the packets are switched through other routes, thankfully the networks have some redundancy.
Circuit VS Packet Switching
Difference between Circuit Switching and Packet Switching
Internet organizations
List of Internet organizations
● IAB (Internet Architecture Board)
● IANA (Internet Assigned Numbers Authority)
● ICANN (Internet Corporation for Assigned Names and Numbers)
● IESG (Internet Engineering Steering Group)
● IETF (Internet Engineering Task Force)
● IRTF (Internet Research Task Force)
● ISOC (Internet Society)
● NANOG (North American Network Operators' Group)
● NRO (Number Resource Organization)
● W3C (World Wide Web Consortium)
● OTF (Open Technology Fund)
● EFF (Electronic Frontier Foundation)
IEEE 802
IEEE 802 is a family of Institute of Electrical and Electronics Engineers (IEEE) standards for local area networks (LAN), personal area
network (PAN), and metropolitan area networks (MAN). The IEEE 802 family of standards has twelve members, numbered 802.1 through
802.12, with a focus group of the LMSC devoted to each.
The services and protocols specified in IEEE 802 map to the lower two layers (data link and physical) of the seven-layer Open Systems
Interconnection (OSI) networking reference model. IEEE 802 divides the OSI data link layer into two sub-layers: logical link control (LLC) and
medium access control (MAC).
Protocol suite
Open Systems Interconnection model -> OSI model
Sockets
A socket is an abstract representation (handle) for the local endpoint of a network communication path.
The Berkeley sockets API represents it as a file descriptor (file handle) that provides a common interface
for input and output to streams of data.
What is a network socket?
BSD -> Berkeley Software Distribution Socket
BSD Sockets are an API for dealing with system sockets, and therefore communication over network.
The API was first introduced in 1983, and still used today.

BSD Sockets are just a set of functions...

● socket() creates a new socket, of some type
● bind() associates a socket with an address
● listen() causes a bound TCP socket to enter listening state
● connect() assigns a free local port number to a socket
● accept() accepts a received incoming attempt to create a new connection,
creates a new socket associated with the socket address pair of this connection
● send(), recv(), sendto(), and recvfrom() are used for sending and receiving data
● close() causes the system to release resources allocated to a socket
● et al.
BSD sockets
BSD Sockets are just a set of functions, type and a workflow.
TCP example:

Client Server
Create a socket Create a socket
connect to some host/port bind to an address
send/receive data Set to listen
close connection accept clients
send/receive data
close connection
TCP / IP
Transmission Control Protocol a communications standard that enables application programs and computing devices to
exchange messages over a network. It is designed to send packets across the internet and ensure the successful delivery
of data and messages over networks.

The Internet Protocol (IP) is the method for sending data from one device to another across the internet. Every device has
an IP address that uniquely identifies it and enables it to communicate with and exchange data with other devices
connected to the internet.

IP is responsible for defining how applications and devices exchange packets of data with each other. It is the principal
communications protocol responsible for the formats and rules for exchanging data and messages between computers on a
single network or several internet-connected networks. It does this through the Internet Protocol Suite (TCP/IP), a group of
communications protocols that are split into four abstraction layers.
OSI vs TCP/IP
 TCP/IP vs TCP and IP
What is the difference between TCP/IP protocol
and TCP model?
TCP/IP and TCP and IP difference?
TCP/IP is a protocol stack which contains different protocols required for the
data transfer from sender to receiver.

TCP - Layer 4 protocol which ensures reliability and is connection oriented.

TCP/IP is also a layered protocol but does not use all of the OSI layers,
though the layers are equivalent in operation and function. The network
access layer is equivalent to OSI layers 1 and 2. The Internet Protocol layer
is comparable to layer 3 in the OSI model. The host-to-host layer is
equivalent to OSI layer 4. These are the TCP and UDP (user datagram
protocol) functions. Finally, the application layer is similar to OSI layers 5, 6,
and 7 combined.

The TCP layer packages the data into packets. A header that’s added to the
data includes source and destination addresses, a sequence number, an
acknowledgment number, a check sum for error detection and correction,
and some other information. The header is 20 octets (octet = 8 bits) grouped
in 32-bit increments.
 TCP Handshake
● Step 1 (SYN) : In the first step, client wants to establish a
connection with server, so it sends a segment with SYN(Synchronize
Sequence Number) which informs server that client is likely to start
communication and with what sequence number it starts segments
with
● Step 2 (SYN + ACK): Server responds to the client request with
SYN-ACK signal bits set. Acknowledgement(ACK) signifies the
response of segment it received and SYN signifies with what
sequence number it is likely to start the segments with
● Step 3 (ACK) : In the final part client acknowledges the response of
server and they both establish a reliable connection with which they
will start the actual data transfer
The steps 1, 2 establish the connection parameter (sequence
number) for one direction and it is acknowledged. The steps 2, 3
establish the connection parameter (sequence number) for the other
direction and it is acknowledged. With these, a full-duplex
communication is established.
TCP Communication
TCP characteristics:
- stream-oriented - data is sent as a stream, thus chunked into packets implicitly, not by the user
- congestion control - a mechanism that ensures that no more packets are sent than the
connection allows. Think a way to quickly get rid of traffic jams.
- in-order data delivery, error-checking, and re-transmission should be self explanatory
How Transmission Control Protocol works
TCP connection is established and maintained until the applications at each end have finished exchanging messages.
TCP performs the following actions:

● determines how to break application data into packets that networks can deliver;
● sends packets to, and accepts packets from, the network layer;
● manages flow control;
● handles retransmission of dropped or garbled packets, as it's meant to provide error-free data transmission
● acknowledges all packets that arrive.

When a web server sends an HTML file to a client, it uses the hypertext transfer protocol (HTTP) to do so. The HTTP program
layer asks the TCP layer to set up the connection and send the file. The TCP stack divides the file into data packets, numbers
them and then forwards them individually to the IP layer for delivery.
Although each packet in the transmission has the same source and destination IP address, packets may be sent along multiple
routes. The TCP program layer in the client computer waits until all packets have arrived. It then acknowledges those it receives
and asks for the retransmission of any it does not, based on missing packet numbers. The TCP layer then assembles the packets
into a file and delivers the file to the receiving application.
TCP/IP Packet Format
When sending packets using TCP/IP, the data portion of each IP packet
is formatted as a TCP segment.

Each TCP segment contains a header and data. The TCP header
contains many more fields than the UDP header and can range in size
from 20 to 60 bytes, depending on the size of the options field.
The TCP header shares some fields with the UDP header: source port
number, destination port number, and checksum. To remember how
those are used, review the UDP article.
Process of TCP packets transmission
Establish connection
When two computers want to send data The SYN and ACK bits are both part of the TCP header:
to each other over TCP, they first need to
establish a connection using a three-way
handshake.

1. The first computer sends a packet

with the SYN bit set to 1 (SYN =
"synchronize?").
2. The second computer sends back a
packet with the ACK bit set to
1(ACK = "acknowledge!") plus the
SYN bit set to 1.
3. The first computer replies back with
In fact, the three packets involved in the three-way
an ACK.
handshake do not typically include any data. Once the
computers are done with the handshake, they're ready
to receive packets containing actual data.
Process of TCP packets transmission
Send packets of data
The sequence and acknowledgement numbers are part
When a packet of data is sent over of the TCP header:
TCP, the recipient must always
acknowledge what they received.
The first computer sends a packet with
data and a sequence number.

The second computer acknowledges it

by setting the ACK bit and increasing
the acknowledgement number by the
length of the received data.

Those two numbers help the computers to keep track of

which data was successfully received, which data was
lost, and which data was accidentally sent twice.
Process of TCP packets transmission
Close the connection
A computer initiates closing the connection by sending a packet with the FIN bit set to 1 (FIN =
finish). The other computer replies with an ACK and another FIN. After one more ACK from the
initiating computer, the connection is closed.
TCP lost packets detection
After sending off a packet, the sender starts a timer and puts the packet in a retransmission queue. If the
timer runs out and the sender has not yet received an ACK from the recipient, it sends the packet again.

The retransmission may lead to the recipient receiving duplicate packets, if a packet was not actually lost
but just very slow to arrive or be acknowledged. If so, the recipient can simply discard duplicate packets.
It's better to have the data twice than not at all!
TCP packets order
Sometimes the missing packet is simply taking a
slower route through the Internet and it arrives soon
after.

Other times, the missing packet may actually be a

lost packet and the sender must retransmit the
packet.

When the recipient sees a higher sequence In both situations, the recipient has to deal with out of
number than what they have acknowledged so order packets. Fortunately, the recipient can use the
far, they know that they are missing at least one
sequence numbers to reassemble the packet data in
the correct order.
packet in between.
In the situation pictured above, the recipient
sees a sequence number of #73 but expected a
sequence number of #37. The recipient lets the
sender know there's something amiss by
sending a packet with an acknowledgement
number set to the expected sequence number.
TCP Error Control
1. Checksum – Every segment contains a checksum field which is used to find corrupted segments. If the segment is
corrupted, then that segment is discarded by the destination TCP and is considered lost.
2. Acknowledgement – TCP has another mechanism called acknowledgement to affirm that the data segments have been
delivered. Control segments that contain no data but have sequence numbers will be acknowledged as well but ACK
segments are not acknowledged.
3. Retransmission – When a segment is missing, delayed to deliver to a receiver, corrupted when it is checked by the
receiver then that segment is retransmitted again. Segments are retransmitted only during two events: when the sender
receives three duplicate acknowledgments (ACK) or when a retransmission timer expires.
1. Retransmission after RTO: TCP always preserves one retransmission time-out (RTO) timer for all sent but not
acknowledged segments. When the timer runs out of time, the earliest segment is retransmitted. Here no timer
is set for acknowledgment. In TCP, the RTO value is dynamic in nature and it is updated using the round trip
time (RTT) of segments. RTT is the time duration needed for a segment to reach the receiver and an
acknowledgment to be received by the sender.
2. Retransmission after Three duplicate ACK segments: RTO method works well when the value of RTO is
small. If it is large, more time is needed to get confirmation about whether a segment has been delivered or not.
Sometimes one segment is lost and the receiver receives so many out-of-order segments that they cannot be
saved. In order to solve this situation, three duplicate acknowledgement method is used and missing segment is
retransmitted immediately instead of retransmitting already delivered segment. This is a fast retransmission
because it makes it possible to quickly retransmit lost segments instead of waiting for timer to end.
TCP Congestion Control
TCP estimates congestion control using congestion window (CWND) and receiver window (RWND)
counters.

Generally congestion control is subset of flow control.

An example of flow control in TCP is sending the RWND from receiver to the sender so it knows how
much data to send to receiver and no more, so as not to overwhelm it.

TCP is using sliding windows for both congestion and flow control.

A TCP cwnd window is the amount of unacknowledged data a sender can send on a particular
connection before it gets an acknowledgment back from the receiver, that it has received some of the
data.

The receiving device should acknowledge each packet it received, indicating the sequence number of
the last well-received packet. After receiving the ACK from the receiving device, the sending device
slides the window to right side.
TCP Congestion Control
TCP not-acknowledges packets by sending
duplicate ACKs of for the last continuous series
of packets. In such situations, all packets
between the not acknowledged one and the
ones already sent are retransmitted.

A solution for this is using so called selective

acknowledgements (SACK) that acknowledge
specific packets.
TCP Additional Info
TCP: Transmission control protocol | Networking tutorial (12 of 13)

TCP Overview Part 1 || TCP Explained || IIT Computer Networks Lecture Series
User Datagram Protocol -> UDP
User Datagram Protocol (UDP) is a Transport Layer protocol. UDP is a
part of the Internet Protocol suite, referred to as UDP/IP suite. Unlike TCP,
it is an unreliable and connectionless protocol. So, there is no need to
establish a connection prior to data transfer.

User datagram protocol (UDP) operates on top of the Internet Protocol

(IP) to transmit datagrams over a network. UDP does not require the
source and destination to establish a three-way handshake before
transmission takes place. Additionally, there is no need for an end-to-end
connection.

Since UDP avoids the overhead associated with connections, error

checks and the retransmission of missing data, it’s suitable for real-time or
high performance applications that don’t require data verification or
correction. If verification is needed, it can be performed at the application
layer.

UDP provides a mechanism to detect corrupt data in packets, but it does

not attempt to solve other problems that arise with packets, such as lost or
out of order packets. That's why UDP is sometimes known as the
Unreliable Data Protocol.
UDP
UDP provides a minimal, unreliable, best-effort, message-passing transport to
applications and upper-layer protocols. Compared to other transport protocols.

UDP does not establish end-to-end connections between communicating end

systems. UDP communication consequently does not incur connection
establishment and teardown overheads and there is minimal associated end
system state. Because of these characteristics, UDP can offer a very efficient
communication transport to some applications, but has no inherent congestion
control or reliability.

One increasingly popular use of UDP is as a tunneling protocol, where a tunnel

endpoint encapsulates the packets of another protocol inside UDP datagrams and
transmits them to another tunnel endpoint, which decapsulates the UDP
datagrams and forwards the original packets contained in the payload. Tunnels
establish virtual links that appear to directly connect locations that are distant in
the physical Internet topology, and can be used to create virtual (private)
networks. Using UDP as a tunneling protocol is attractive when the payload
protocol is not supported by middleboxes that may exist along the path, because
many middleboxes support UDP transmissions.

UDP does not provide any communications security. Applications that need to
protect their communications against eavesdropping, tampering, or message
forgery therefore need to separately provide security services using additional
protocol mechanisms.
UDP
Application designers are generally aware that UDP does not
provide any reliability, e.g., it does not retransmit any lost packets.
Often, this is a main reason to consider UDP as a transport.
Applications that do require reliable message delivery therefore need
to implement appropriate protocol mechanisms in their applications

UDP's best effort service does not protect against datagram

duplication, i.e., an application may receive multiple copies of the
same UDP datagram. Application designers therefore need to verify
that their application gracefully handles datagram duplication and
may need to implement mechanisms to detect duplicates.

The Internet may also significantly delay some packets with respect
to others, e.g., due to routing transients, intermittent connectivity, or
mobility. This can cause reordering, where UDP datagrams arrive at
the receiver in an order different from the transmission order.
Applications that require ordered delivery must restore datagram
ordering themselves.
UDP benefits and downsides
UDP has a number of benefits for different types of applications, including:

● No retransmission delays – UDP is suitable for time-sensitive applications that can’t afford retransmission delays for
dropped packets. Examples include Voice over IP (VoIP), online games, and media streaming.
● Speed – UDP’s speed makes it useful for query-response protocols such as DNS, in which data packets are small
and transactional.
● Suitable for broadcasts – UDP’s lack of end-to-end communication makes it suitable for broadcasts, in which
transmitted data packets are addressed as receivable by all devices on the internet. UDP broadcasts can be
received by large numbers of clients without server-side overhead.

At the same time, UDP’s lack of connection requirements and data verification can create a number of issues when
transmitting packets. These include:

● No guaranteed ordering of packets.

● No verification of the readiness of the computer receiving the message.
● No protection against duplicate packets.
● No guarantee the destination will receive all transmitted bytes. UDP, however, does provide a checksum to verify
individual packet integrity.
UDP Packet Format

When sending packets using UDP over IP, the data portion of each IP packet is formatted as a
UDP segment.

Each UDP segment contains an 8-byte header and variable length data.
UDP Packet Format
Port numbers
The first four bytes of the UDP header store the port numbers for the source and destination.

A networked device can receive messages on different virtual ports, similar to how an ocean harbor can receive boats on
different ports. The different ports help distinguish different types of network traffic.

Segment Length

The next two bytes of the UDP header store the length (in bytes) of the segment (including the header).

Two bytes is 161616 bits, so the length can be as high as this binary number:

111111111111111111111111111111111111111111111111

In decimal, that's (2^16 - 1) or 65535. Thus, the maximum length of a UDP segment is 65535 bytes.
 UDP Packet Format
Checksum
The final two bytes of the UDP header is the checksum, a field that's
The entire UDP segment could look like
used by the sender and receiver to check for data corruption. this:

Before sending off the segment, the sender:

1. Computes the checksum based on the data in the segment.

2. Stores the computed checksum in the field.

Upon receiving the segment, the recipient:

1. Computes the checksum based on the received segment.

2. Compares the checksums to each other. If the checksums aren't
equal, it knows the data was corrupted.
 TCP vs UDP
TCP is a connection-oriented protocol and requires handshaking to set up end-to-end communications. Once a
connection is set up, user data may be sent bi-directionally over the connection.
● Reliable – TCP manages message acknowledgment, retransmission and timeouts. Multiple attempts to
deliver the message are made. If data gets lost along the way, data will be re-sent. In TCP, there's either
no missing data, or, in case of multiple timeouts, the connection is dropped.
● Ordered – If two messages are sent over a connection in sequence, the first message will reach the
receiving application first. When data segments arrive in the wrong order, TCP buffers the out-of-order
data until all data can be properly re-ordered and delivered to the application.
● Heavyweight – TCP requires three packets to set up a socket connection, before any user data can be
sent. TCP handles reliability and congestion control.
● Streaming – Data is read as a byte stream, no distinguishing indications are transmitted to signal
message (segment) boundaries.
User Datagram Protocol is a simpler message-based connectionless protocol. Connectionless protocols do not set up
a dedicated end-to-end connection. Communication is achieved by transmitting information in one direction from
source to destination without verifying the readiness or state of the receiver.
● Unreliable – When a UDP message is sent, it cannot be known if it will reach its destination; it could get
lost along the way. There is no concept of acknowledgment, retransmission, or timeout.
● Not ordered – If two messages are sent to the same recipient, the order in which they arrive cannot be
guaranteed.
● Lightweight – There is no ordering of messages, no tracking connections, etc. It is a very simple
transport layer designed on top of IP.
● Datagrams – Packets are sent individually and are checked for integrity on arrival. Packets have definite
boundaries which are honored upon receipt; a read operation at the receiver socket will yield an entire
message as it was originally sent.
● No congestion control – UDP itself does not avoid congestion. Congestion control measures must be
implemented at the application level or in the network.
● Broadcasts – being connectionless, UDP can broadcast - sent packets can be addressed to be
receivable by all devices on the subnet.
● Multicast – a multicast mode of operation is supported whereby a single datagram packet can be
automatically routed without duplication to a group of subscribers.
TCP vs UDP

TCP vs UDP For Video Streaming

TCP vs UDP
 TCP vs UDP speed
The reason for UDP’s superior speed over TCP is that its non-existent ‘acknowledgement’
supports a continuous packet stream. Since TCP connection always acknowledges a set
of packets (whether or not the connection is totally reliable), a retransmission must occur
for every negative acknowledgement where a data packet has been lost.

But because UDP avoids the unnecessary overheads of TCP transport, it’s incredibly
efficient in terms of bandwidth, and much less demanding of poor performing networks, as
well.
Email Protocols
There are 3 main types of email protocols:

1. SMTP (Simple Mail Transfer Protocol),

2. POP (Post Office Protocol)
3. IMAP (Internet Message Access Protocol).
Email Protocols
There are three common protocols used to deliver email over
the Internet: the Simple Mail Transfer Protocol (SMTP), the
Post Office Protocol (POP), and the Internet Message Access
Protocol (IMAP).
All three use TCP, and the last two are used for accessing
electronic mailboxes. Special records stored in DNS servers
play a role as well, using UDP. The current version of POP is
version 3 (POP3) and the current version of IMAP is version 4
(IMAP4).
Although not a protocol, there is a series of Multipurpose
Internet Mail Extensions (just MIME, never “MIMEs”) for
various types of email attachments (not just simple text).
Finally, a number of related specifications add authentication to
the basic email protocols.
SMTP -> Simple Mail Transfer Protocol
Most internet systems use SMTP as a method to transfer mail from one user to another. SMTP is a push protocol
and is used to send the mail whereas POP (post office protocol) or IMAP (internet message access protocol) are
used to retrieve those emails at the receiver’s side.

The main work of SMTP is to send emails. It can be used by the client to send emails to the server or used by one server to send
emails to another server. Precisely, in an SMTP scenario, the server decides the operations it wants to execute/operate. The
other server authorizes the operation and this is how the message is sent.

Ports on which SMTP protocol works:

● Port 25:- This is the default port for SMTP and it is non-encrypted.
● Port 2525:- In case, port 25 is filtered and the user sends non-encrypted emails.
● Port 465:- This port of SMTP is used for establishing a secure connection

SMTP is an application layer protocol. The client who wants to send the mail opens a TCP connection to the
SMTP server and then sends the mail across the connection. The SMTP server is an always-on listening mode.
As soon as it listens for a TCP connection from any client, the SMTP process initiates a connection through port
25. After successfully establishing a TCP connection the client process sends the mail instantly.
SMTP
In the SMTP model user deals with the user agent (UA), for example, Microsoft Outlook, Netscape, Mozilla, etc. In
order to exchange the mail using TCP, MTA (Message transfer agent) is used. The user sending the mail doesn’t
have to deal with MTA as it is the responsibility of the system admin to set up a local MTA.
An MTA is just an element of the email delivery process. It receives an email from the mail/message submission
agent (MSA), which, in turn, receives it from the mail user agent (MUA). The MUA is commonly known as an email
client – an app you use to handle the email-related stuff.
The MTA maintains a small queue of mails so that it can schedule repeat delivery of mails in case the receiver is
not available. The MTA delivers the mail to the mailboxes and the information can later be downloaded by the user
agents.
POP3 -> Post Office Protocol V3
Represents Post Office Protocol version 3. It is used to download emails from the remote server to the local client. POP
allows us to download email messages from a local computer and delete emails from the server. This leads to less space
usage on the server. Although it supports offline email processing, it lacks advanced features like contacts, etc. It only
downloads new emails which have been received.

Ports on which POP3 protocol works:

● Port 110:- This is the default port for POP3 and it is non-encrypted.
● Port 995:- This port of POP3 is used for establishing a secure connection.

Key Points

● POP is an application layer internet standard protocol.

● Since POP supports offline access to the messages, thus requires less internet usage time.
● POP does not allow search facility.
● In order to access the messaged, it is necessary to download them.
● It allows only one mailbox to be created on server.
● It is not suitable for accessing non mail data.
● POP commands are generally abbreviated into codes of three or four letters. Eg. STAT.
POP3
mail clients that use POP3 store messages on the user’s computer, deleting them from the email server. People using
email clients with POP3 also have the option of keeping their emails on the server after download.

This email protocol was designed to enable users to read, draft, and delete emails when there is no smooth access to the
internet. With POP3, emails are delivered as soon as the user connects to the internet.
 IMAP -> Internet Message Access Protocol
IMAP is used to receive emails from a server. IMAP settings have the option of storing emails on the mail server. It provides
an edge for the user to access emails from multiple locations, and it also allows simultaneous access by multiple clients.
This is why IMAP is more apt if the user accesses his/her email from different locations or if the messages of the user are
handled by multiple people.

The end-user can delete or allow the post as it is in their inbox after going through emails. Offline mode is also supported
for this setup, which means that even when offline changes are visible after reconnection. To understand it accurately, a
copy of recent emails and old emails will be available online.

Ports on which IMAP protocol works:

● Port 143:- This is the default port for IMAP and it is non-encrypted.
● Port 993:- This port of IMAP is used for establishing a secure connection.
IMAP
IMAP features:

● It is capable of managing multiple mailboxes and organizing them into various categories.
● Provides adding of message flags to keep track of which messages are being seen.
● It is capable of deciding whether to retrieve email from a mail server before downloading.
● It makes it easy to download media when multiple files are attached.

Advantages :

● It offers synchronization across all the maintained sessions by the user.

● It provides security over POP3 protocol as the email only exists on the IMAP server.
● Users have remote access to all the contents.
● It offers easy migration between the devices as it is synchronized by a centralized server.
● There is no need to physically allocate any storage to save contents.

Disadvantages :

● IMAP is complex to maintain.

● Emails of the user are only available when there is an internet connection.
● It is slower to load messages.
● Some emails don’t support IMAP which makes it difficult to manage.
● Many browser-based solutions are unavailable due to not support of IMAP.
SMTP vs POP3 vs IMAP
FTP
File Transfer Protocol (FTP) is a
communications protocol used for transferring or
exchanging files between two computers.
File Transfer Protocol is the Internet standard for
transferring or exchanging files from one
computer to another using TCP or IP networks.
 FTP -> File Transfer Protocol
File Transfer Protocol is an application layer protocol
which moves files between local and remote file
systems. It runs on the top of TCP, like HTTP. To
transfer a file, 2 TCP connections are used by FTP in
parallel: control connection and data connection.

What is control connection?

For sending control information like user identification,
password, commands to change the remote directory,
commands to retrieve and store files, etc., FTP makes
use of control connection. The control connection is
initiated on port number 21.
What is data connection?
For sending the actual file, FTP makes use of data
connection. A data connection is initiated on port
number 20.
FTP sends the control information out-of-band as it
uses a separate control connection. Some protocols
send their request and response header lines and the
data in the same TCP connection. For this reason, they
are said to send their control information in-band. HTTP
and SMTP are such examples.
FTP session
FTP Session :

When a FTP session is started between a client and a

server, the client initiates a control TCP connection with
the server side. The client sends control information over
this. When the server receives this, it initiates a data
connection to the client side. Only one file can be sent
over one data connection. But the control connection
remains active throughout the user session. As we know
HTTP is stateless i.e. it does not have to keep track of any
user state. But FTP needs to maintain a state about its
user throughout the session.
 FTP Commands & Replies
FTP Commands – Some of the FTP commands are : FTP Replies – Some of the FTP replies are :

USER – This command sends the user identification to the server. 200 Command okay.

PASS – This command sends the user password to the server. 530 Not logged in.

CWD – This command allows the user to work with a different directory or dataset for file 331 User name okay, need a password.
storage or retrieval without altering his login or accounting information.
225 Data connection open; no transfer in progress.
RMD – This command causes the directory specified in the path-name to be removed as
221 Service closing control connection.
a directory.
551 Requested action aborted: page type unknown.
MKD – This command causes the directory specified in the pathname to be created as a
directory. 502 Command not implemented.
PWD – This command causes the name of the current working directory to be returned in 503 Bad sequence of commands.
the reply.
504 Command not implemented for that parameter.
RETR – This command causes the remote host to initiate a data connection and to send
the requested file over the data connection.

STOR – This command causes to store a file into the current directory of the remote host.

LIST – Sends a request to display the list of all the files present in the directory.

ABOR – This command tells the server to abort the previous FTP service command and
any associated transfer of data.

QUIT – This command terminates a USER and if file transfer is not in progress, the server
closes the control connection.
FTP -> Passive vs Active
FTP sessions work in active or passive modes:

● Active mode. After a client initiates a

session via a command channel request,
the server creates a data connection back
to the client and begins transferring data.
● Passive mode. The server uses the
command channel to send the client the
information it needs to open a data
channel. Because passive mode has the
client initiating all connections, it works
well across firewalls and network address
translation gateways.
Stream Control Transmission Protocol
SCTP is a connection-oriented protocol, similar to TCP, but provides message-oriented data transfer, similar to UDP

SCTP applications submit data for transmission in messages (groups of bytes) to the SCTP transport layer. SCTP places
messages and control information into separate chunks (data chunks and control chunks), each identified by a chunk
header. The protocol can fragment a message into multiple data chunks, but each data chunk contains data from only one
user message. SCTP bundles the chunks into SCTP packets. The SCTP packet, which is submitted to the Internet Protocol,
consists of a packet header, SCTP control chunks (when necessary), followed by SCTP data chunks (when available).
SCTP
In general, SCTP may provide more flexibility for certain applications, like Voice over IP (VoIP), that require the reliable but
message-oriented data transfer. For this category of applications, SCTP is most likely better-suited than TCP or UDP.

● TCP provides reliable and strict order-of-transmission data delivery. For applications that need reliability, but can
tolerate unordered or partially ordered data delivery, TCP may cause some unnecessary delay because of
head-of-line blocking. With the concept of multiple streams within a single connection, SCTP can provide strictly
ordered delivery within a stream while logically isolating data from different streams.
● SCTP is message-oriented, unlike TCP, which is byte-oriented. Because of the byte-oriented nature of TCP, the
application has to add its own record marking to maintain message boundaries.
● SCTP provides some degree of fault tolerance by using the Multihoming feature. A host is considered multihomed
when it has more than one network interface attached, either on the same or different networks. An SCTP
association can be established between two multihomed hosts. In this case, all IP addresses of both endpoints are
exchanged at association startup; this allows each endpoint to use any of these addresses over the life of the
connection if one of the interfaces is down for any reason, as long as the peer is reachable through the alternate
interfaces.
SCTP Conceptual Model
HTTP -> Hypertext Transfer Protocol
The HTTP is high level, textual, application layer protocol
It was designed for communication between web browsers and web servers, but it can also be used for other purposes.
HTTP follows a classical client-server model, with a client opening a connection to make a request, then waiting until it
receives a response. HTTP is a stateless protocol, meaning that the server does not keep any data (state) between two
requests.
HTTP Basics
Clients and servers communicate by exchanging individual messages (as opposed to a stream of data). The messages
sent by the client are called requests and the messages sent by the server as an answer are called responses.

Designed in the early 1990s, HTTP is an extensible protocol which has evolved over time. It is an application layer protocol
that is sent over TCP, or over a TLS-encrypted TCP connection, though any reliable transport protocol could theoretically be
used. Due to its extensibility, it is used to not only fetch hypertext documents, but also images and videos or to post content
to servers, like with HTML form results. HTTP can also be used to fetch parts of documents to update Web pages on
demand.
Basic aspects of HTTP
HTTP is simple
HTTP is generally designed to be simple and human readable, even with the added complexity introduced in HTTP/2 by
encapsulating HTTP messages into frames. HTTP messages can be read and understood by humans, providing easier
testing for developers, and reduced complexity for newcomers.

HTTP is extensible
Introduced in HTTP/1.0, HTTP headers make this protocol easy to extend and experiment with. New functionality can even
be introduced by a simple agreement between a client and a server about a new header's semantics.

HTTP is stateless, but not sessionless

HTTP is stateless: there is no link between two requests being successively carried out on the same connection. This
immediately has the prospect of being problematic for users attempting to interact with certain pages coherently, for
example, using e-commerce shopping baskets. But while the core of HTTP itself is stateless, HTTP cookies allow the use of
stateful sessions. Using header extensibility, HTTP Cookies are added to the workflow, allowing session creation on each
HTTP request to share the same context, or the same state.
HTTP connections
Before a client and server can exchange an HTTP
request/response pair, they must establish a TCP connection,
a process which requires several round-trips. The default
behavior of HTTP/1.0 is to open a separate TCP connection
for each HTTP request/response pair. This is less efficient
than sharing a single TCP connection when multiple requests
are sent in close succession.

In order to mitigate this flaw, HTTP/1.1 introduced pipelining

(which proved difficult to implement) and persistent
connections: the underlying TCP connection can be partially
controlled using the Connection header. HTTP/2 went a
step further by multiplexing messages over a single
connection, helping keep the connection warm and more
efficient.
HTTP Flow
When a client wants to communicate with a server, either the final server or an intermediate proxy, it performs the following
steps:

1. Open a TCP connection: The TCP connection is used to send a request, or several, and receive an answer. The
client may open a new connection, reuse an existing connection, or open several TCP connections to the servers.
2. Send an HTTP message: HTTP messages (before HTTP/2) are human-readable. With HTTP/2, these simple
messages are encapsulated in frames, making them impossible to read directly, but the principle remains the same.
For example:

3. Read the response sent by the server, such as:

HTTP control -> What can be controlled by HTTP
● Caching How documents are cached can be controlled by HTTP. The server can instruct proxies and clients about
what to cache and for how long. The client can instruct intermediate cache proxies to ignore the stored document.
● Relaxing the origin constraint To prevent snooping and other privacy invasions, Web browsers enforce strict
separation between Web sites. Only pages from the same origin can access all the information of a Web page.
Though such a constraint is a burden to the server, HTTP headers can relax this strict separation on the server side,
allowing a document to become a patchwork of information sourced from different domains; there could even be
security-related reasons to do so.
● Authentication Some pages may be protected so that only specific users can access them. Basic authentication may
be provided by HTTP, either using the WWW-Authenticate and similar headers, or by setting a specific session
using HTTP cookies.
● Proxy and tunneling Servers or clients are often located on intranets and hide their true IP address from other
computers. HTTP requests then go through proxies to cross this network barrier. Not all proxies are HTTP proxies.
The SOCKS protocol, for example, operates at a lower level. Other protocols, like ftp, can be handled by these
proxies.
● Sessions Using HTTP cookies allows you to link requests with the state of the server. This creates sessions, despite
basic HTTP being a state-less protocol. This is useful not only for e-commerce shopping baskets, but also for any
site allowing user configuration of the output.
HTTP Messages
HTTP messages are how data is exchanged between a server and a client.
There are two types of messages:
1. requests sent by the client to trigger an action on the server
2. responses, the answer from the server.

HTTP messages are composed of textual information encoded in ASCII, and span over multiple lines. In HTTP/1.1, and
earlier versions of the protocol, these messages were openly sent across the connection. In HTTP/2, the once
human-readable message is now divided up into HTTP frames, providing optimization and performance improvements.
HTTP headers
HTTP headers let the client and the server pass additional information with an HTTP request or response.
An HTTP header consists of its case-insensitive name followed by a colon (:), then by its value. Whitespace before the
value is ignored.

EX: Content-Type: application/json

Custom proprietary headers have historically been used with an X- prefix, but this convention was deprecated in June 2012
because of the inconveniences it caused when nonstandard fields became standard in RFC 6648

Full list of HTTP headers: HTTP headers - HTTP | MDN

Headers can be grouped according to their contexts:

● Request headers contain more information about the resource to be fetched, or about the client requesting the
resource.
● Response headers hold additional information about the response, like its location or about the server providing it.
● Representation headers contain information about the body of the resource, like its MIME type, or
encoding/compression applied.
● Payload headers contain representation-independent information about payload data, including content length and
the encoding used for transport.
HTTP headers
First we’ll look at the 5 headers that you will most often interact with:

1. Authorization: This client-side header communicates the authorization status of the user. This is a token with the authorization
status of the user. It is different from authentication, which is concerned with the actual validation of the user's identity. The
authorization header is generally used to send a token that the server can process to determine if the user is allowed to access
the given resource.

2. Content-Type: This client and server-side header tells the recipient of the request what the body of your request contains. For
example, is the data type JSON, text, etc...

3. Accept: This Client-side header determines the data type that you expect to get back from the server, and thus which your
application is expected to be able to understand. It can be useful to provide this information to the server so that it won’t waste
time processing a request from a client that can’t receive a given data type.

4. Cookie: This client-side header contains other metadata stored in cookies on the users browser. these are often, but not
exclusively, originally set by the server with the Set-Cookie header, and can be used to communicate any type of metadata
about the user, including browsing history or login status.

5. Access-Control-Allow-Origin: This is a server-side header. It contains information about the servers accepted web addresses,
as per a protocol called CORs. Essentially it is the servers built in whitelist of clients that it expects to interact with. If you see an
error referencing “Cross-origin-request”s in your client side code, it likely has to do with this header. For more information on
CORS I recommend this article.
HTTP request methods
HTTP defines a set of request methods to indicate the desired action to be performed for a given resource. Although they
can also be nouns, these request methods are sometimes referred to as HTTP verbs.
Each of them implements a different semantic, but some common features are shared by a group of them: e.g. a request
method can be safe, idempotent, or cacheable.

GET
PUT
The GET method requests a representation of the
The PUT method replaces all current
specified resource. Requests using GET should only
representations of the target resource with the
retrieve data.
request payload.

HEAD
DELETE
The HEAD method asks for a response identical to a
The DELETE method deletes the specified
GET request, but without the response body.
resource.

POST
CONNECT
The POST method submits an entity to the specified
The CONNECT method establishes a tunnel to
resource, often causing a change in state or side
the server identified by the target resource.
effects on the server.
HTTP request methods idempotency -> POST VS PUT
An HTTP method is idempotent if an identical request can be made once or several times in a row with the same effect
while leaving the server in the same state. In other words, an idempotent method should not have any side-effects (except
for keeping statistics). Implemented correctly, the GET, HEAD, PUT, and DELETE methods are idempotent, but not the POST
method.

OPTIONS
The OPTIONS method describes the
To be idempotent, only the actual back-end state of the server is
communication options for the target resource.
considered, the status code returned by each request may differ: the
first call of a DELETE will likely return a 200, while successive ones
will likely return a 404. Another implication of DELETE being
TRACE idempotent is that developers should not implement RESTful APIs
The TRACE method performs a message with a delete last entry functionality using the DELETE method.
loop-back test along the path to the target
Note that the idempotence of a method is not guaranteed by the
resource. server and some applications may incorrectly break the idempotence
constraint.

PATCH
The PATCH method applies partial modifications
to a resource.
HTTP response status codes
HTTP response status codes indicate whether a specific HTTP request has been
successfully completed. Responses are grouped in five classes:

1. Informational responses (100–199)

2. Successful responses (200–299)
3. Redirection messages (300–399)
4. Client error responses (400–499)
5. Server error responses (500–599)

Full list of HTTP status code: HTTP response status codes - HTTP | MDN
HTTP Message Structure
HTTP requests, and responses, share similar structure and are composed of:

1. A start-line describing the requests to be implemented, or its status of whether successful or a failure. This start-line is always a
single line.
2. An optional set of HTTP headers specifying the request, or describing the body included in the message.
3. A blank line indicating all meta-information for the request has been sent.
4. An optional body containing data associated with the request (like content of an HTML form), or the document associated with a
response. The presence of the body and its size is specified by the start-line and HTTP headers.

The start-line and HTTP headers of the HTTP message are collectively known as the head of the requests, whereas its payload is
known as the body.
HTTP Message Start Line
The start line of an HTTP request contain three elements:

1. An HTTP method, a verb (like GET, PUT or POST) or a noun (like HEAD or OPTIONS), that describes the action to
be performed. For example, GET indicates that a resource should be fetched or POST means that data is pushed to
the server (creating or modifying a resource, or generating a temporary document to send back).
2. The request target, usually a URL, or the absolute path of the protocol, port, and domain are usually characterized by
the request context. The format of this request target varies between different HTTP methods.
3. The HTTP version, which defines the structure of the remaining message, acting as an indicator of the expected
version to use for the response.

The start line of an HTTP response, called the status line, contains the following information:

1. The protocol version, usually HTTP/1.1.

2. A status code, indicating success or failure of the request. Common status codes are 200, 404, or 302
3. A status text. A brief, purely informational, textual description of the status code to help a human understand the
HTTP message.

A typical status line looks like: HTTP/1.1 404 Not Found.

HTTP Request Headers
Many different headers can appear in requests. They can be divided in several groups:

● General headers, like Via, apply to the message as a whole.

● Request headers, like User-Agent or Accept, modify the request by specifying it further (like
Accept-Language), by giving context (like Referer), or by conditionally restricting it (like If-None).
● Representation headers like Content-Type that describe the original format of the message data and any encoding
applied (only present if the message has a body).
HTTP Response Headers
Many different headers can appear in responses. These can be divided into several groups:

● General headers, like Via, apply to the whole message.

● Response headers, like Vary and Accept-Ranges, give additional information about the server which doesn't fit in
the status line.
● Representation headers like Content-Type that describe the original format of the message data and any encoding
applied (only present if the message has a body).
HTTP Message Body
The final part of the request is its body. Not all requests have one: requests fetching resources, like GET, HEAD, DELETE, or
OPTIONS, usually don't need one. Some requests send data to the server in order to update it: as often the case with POST
requests (containing HTML form data).

Bodies can be broadly divided into two categories:

● Single-resource bodies, consisting of one single file, defined by the two headers: Content-Type and
Content-Length.
● Multiple-resource bodies, consisting of a multipart body, each containing a different bit of information. This is typically
associated with HTML Forms.

The last part of a response is the body. Not all responses have one: responses with a status code that sufficiently answers
the request without the need for corresponding payload (like 201 Created or 204 No Content) usually don't.

Bodies can be broadly divided into three categories:

● Single-resource bodies, consisting of a single file of known length, defined by the two headers: Content-Type and
Content-Length.
● Single-resource bodies, consisting of a single file of unknown length, encoded by chunks with Transfer-Encoding
set to chunked.
● Multiple-resource bodies, consisting of a multipart body, each containing a different section of information. These are
relatively rare.
HTTP/2 Frames
HTTP/1.x messages have a few drawbacks for
performance:

● Headers, unlike bodies, are uncompressed.

● Headers are often very similar from one
message to the next one, yet still repeated
across connections.
● No multiplexing can be done. Several
connections need opening on the same server:
and warm TCP connections are more efficient
than cold ones.

HTTP/2 introduces an extra step: it divides HTTP/1.x

messages into frames which are embedded in a stream.
Data and header frames are separated, which allows
header compression. Several streams can be combined
together, a process called multiplexing, allowing more
efficient use of underlying TCP connections.
Connection management in HTTP
Connection management in
HTTP/1.x
In HTTP/1.x, there are several models:

● short-lived connections,
● persistent connections
● HTTP pipelining.

Initially HTTP used a single model to handle

such connections. These connections were
short-lived: a new one created each time a
request needed sending, and closed once the
answer had been received.

Two newer models were created in HTTP/1.1.

The persistent-connection model keeps
connections opened between successive
requests, reducing the time needed to open
new connections. The HTTP pipelining model
goes one step further, by sending several
successive requests without even waiting for an
answer, reducing much of the latency in the
network.
Connection management in
HTTP. Short lived connection
Each HTTP request is completed on its own connection; this means
a TCP handshake happens before each HTTP request, and these
are serialized.

The TCP handshake itself is time-consuming, but a TCP connection

adapts to its load, becoming more efficient with more sustained (or
warm) connections. Short-lived connections do not make use of this
efficiency feature of TCP, and performance degrades from optimum
by persisting to transmit over a new, cold connection.

This model is the default model used in HTTP/1.0 (if there is no

Connection header, or if its value is set to close). In HTTP/1.1,
this model is only used when the Connection header is sent with a
value of close.

Fetching HTML and CSS via separate TCP connections

Connection management in HTTP. Persistent connection
Short-lived connections have two major hitches: the time taken
to establish a new connection is significant, and performance
of the underlying TCP connection gets better only when this
connection has been in use for some time (warm connection).
To ease these problems, the concept of a persistent
connection has been designed, even prior to HTTP/1.1.
Alternatively this may be called a keep-alive connection.

A persistent connection is one which remains open for a

period of time, and can be reused for several requests, saving
the need for a new TCP handshake, and utilizing TCP's
performance enhancing capabilities. This connection will not
stay open forever: idle connections are closed after some time
(a server may use the Keep-Alive header to specify a
minimum time the connection should be kept open).

HTTP/1.0 connections are not persistent by default. Setting

Connection to anything other than close, usually
retry-after, will make them persistent.

In HTTP/1.1, persistence is the default, and the header is no

longer needed (but it is often added as a defensive measure
against cases requiring a fallback to HTTP/1.0).
Fetching HTML and CSS with connection keepalive
Connection management in HTTP. HTTP pipelining
By default, HTTP requests are issued sequentially. The next
request is only issued once the response to the current
request has been received. As they are affected by network
latencies and bandwidth limitations, this can result in
significant delay before the next request is seen by the
server.

Pipelining is the process to send successive requests, over

the same persistent connection, without waiting for the
answer. This avoids latency of the connection. Theoretically,
performance could also be improved if two HTTP requests
were to be packed into the same TCP message. The typical
MSS (Maximum Segment Size), is big enough to contain
several simple requests, although the demand in size of
HTTP requests continues to grow.

Not all types of HTTP requests can be pipelined: only

idempotent methods, that is GET, HEAD, PUT and DELETE,
can be replayed safely. Should a failure happen, the pipeline
content can be repeated.

Today, every HTTP/1.1-compliant proxy and server should

support pipelining, though many have limitations in practice:
a significant reason no modern browser activates this feature
by default.
HTTP Long-Polling
With Long-Polling, the client requests information from the
server with the expectation that the server may not respond
immediately.

● If the server does not have any data available for the
client, instead of sending an empty response, the server
holds the request and waits until some data becomes
available.
● Once the data becomes available, a full response is sent
to the client. The client then immediately re-request
information from the server so that the server will almost
always have an available waiting request that it can use
to deliver data in response to an event.
4. The client receives the response.
The Long-Polling process consists of the following steps:
5. The client usually makes a new request right away or after
1. The client makes a request to the server. some defined interval so that the connection with the server is
2. The server receives the request and delays sending established again.
anything to the client until the requested data is available
6. Each Long-Poll request has a timeout. The client has to
or there is any other update related to data.
reconnect periodically after the connection is closed due to
3. When the data is available, the response is sent to the
timeouts.
client.
Server Sent Events -> SSE
SSE is a way of establishing long-term
communication between client and server that
enables the server to proactively push data to the
client.

It is unidirectional: once the client sends the request

it can only receive the responses without the ability
to send new requests over the same connection.

SSE process consists of the following steps:

1. The client makes a request for some data from the

server.
2. The connection between client and server is being
established and it remains open.
3. The server sends responses/events to the client
every time a new data or an update about requested
data is available.
WebSockets
The idea of WebSockets was borne out of the limitations of
HTTP-based technology. With HTTP, a client requests a resource,
and the server responds with the requested data. HTTP is a strictly
unidirectional protocol — any data sent from the server to the client
must be first requested by the client. Long-polling has traditionally
acted as a workaround for this limitation. With long-polling, a client
makes an HTTP request with a long timeout period, and the server
uses that long timeout to push data to the client. Long-polling works,
but comes with a drawback — resources on the server are tied up
throughout the length of the long-poll, even when no data is available
to send.

WebSockets, on the other hand, allow for sending message-based

data, similar to UDP, but with the reliability of TCP. WebSocket uses
HTTP as the initial transport mechanism, but keeps the TCP
connection alive after the HTTP response is received so that it can
be used for sending messages between client and server.
WebSockets allow us to build “real-time” applications without the use
of long-polling.
WebSockets
The protocol consists of an opening handshake followed by basic
message framing, layered over TCP.

WebSockets begin life as a standard HTTP request and response.

Within that request response chain, the client asks to open a
WebSocket connection, and the server responds (if its able to). If this
initial handshake is successful, the client and server have agreed to
use the existing TCP/IP connection that was established for the
HTTP request as a WebSocket connection. Data can now flow over
this connection using a basic framed message protocol. Once both
parties acknowledge that the WebSocket connection should be
closed, the TCP connection is torn down.
WebSocket connection upgrade (handshake)
Protocol upgrade mechanism - HTTP | MDN

A WebSocket server is nothing more than an application listening on any port of a TCP server that follows a specific
protocol. After creating the initial HTTP/1.1 session, you need to request the upgrade by adding to a standard request the
Upgrade and Connection headers. The client will send a pretty standard HTTP request with headers that looks like this
(the HTTP version must be 1.1 or greater, and the method must be GET):

The Upgrade header field is used by clients to invite the server

to switch to one of the listed protocols, in descending
preference order.
WebSocket upgrades allow additional headers to configure
details about the WebSocket connection as well as to offer a
degree of security in opening the connection.

If the server decides to upgrade the connection, it sends back a 101 Switching Protocols response status with an
Upgrade header that specifies the protocol(s) being switched to. If it does not (or cannot) upgrade the connection, it ignores the
Upgrade header and sends back a regular response (for example, a 200 OK).

Right after sending the 101 status code, the server can begin speaking the new protocol, performing any additional
protocol-specific handshakes as necessary. Effectively, the connection becomes a two-way pipe as soon as the upgraded
response is complete, and the request that initiated the upgrade can be completed over the new protocol.
WebSocket connection upgrade (handshake)
When the server receives the handshake request, it
should send back a special response that indicates
that the protocol will be changing from HTTP to
WebSocket. That header looks something like the
following

This doesn't directly relate to the WebSocket protocol, but it's worth mentioning here: your server must keep track of clients'
sockets so you don't keep handshaking again with clients who have already completed the handshake. The same client IP
address can try to connect multiple times. However, the server can deny them if they attempt too many connections in order
to save itself from Denial-of-Service attacks.

For example, you might keep a table of usernames or ID numbers along with the corresponding WebSocket and other data
that you need to associate with that connection.

Usually all this things are done by websockets implementation libraries, but if you decide to implement some custom server
from scratch you will have keep track of clients.
WebSocket Messages
WebSocket is a framed protocol, meaning that a chunk of data (a message) is divided into a number of discrete chunks,
with the size of the chunk encoded in the frame. The frame includes a frame type, a payload length, and a data portion. An
overview of the frame is given in RFC 6455 and reproduced here.

The most important bits so that we can gain an

understanding of the WebSocket protocol.

Fin Bit
The first bit of the WebSocket header is the Fin bit.
This bit is set if this frame is the last data to complete
this message.

Payload data
The Payload data includes arbitrary application data
and any extension data that has been negotiated
between the client and the server. Extensions are
negotiated during the initial handshake and allow
you to extend the WebSocket protocol for additional
uses.
WebSocket Messages

To close a WebSocket connection, a closing frame is sent (opcode 0x08). In addition to the opcode, the close frame may
contain a body that indicates the reason for closing. If either side of a connection receives a close frame, it must send a
close frame in response, and no more data should be sent over the connection. Once the close frame has been received by
both parties, the TCP connection is torn down. The server always initiates closing the TCP connection.
 HTTP Polling vs SSE vs WebSocket

Advantages of SSE over Websockets:

● Transported over simple HTTP instead of a
custom protocol
● Built in support for re-connection and event-id

Advantages of Websockets over SSE:

● Real time, two directional communication.
● Native support in more platforms
 API Architectures
In order to integrate applications quickly and at scale, APIs are
realized using protocols and/or specifications to define the
semantics and syntax of the messages passed across the wire.
These specifications make up the API architecture.

Over time, different API architectural styles have been released.

Each of them has its own patterns of standardizing data
exchange. A pull of choices raises endless debates as to which
architectural style is best.

Comparing API Architectural Styles: SOAP vs REST vs GraphQL

vs RPC
Representational State Transfer -> REST
The most common API style today was originally described in
2000 by Roy Fielding in his doctoral dissertation. REST makes
server-side data available representing it in simple formats, often
JSON and XML.

RESTful architecture should comply with six architectural

constraints:

● uniform interface: permitting a uniform way of interacting

with a given server regardless of device or application
type
● stateless: the necessary state to handle the request as
contained within the request itself and without the server
storing anything related to the session
● caching
● client-server architecture: allowing for independent
evolution of either side
● layered system of the application In REST, things are done using HTTP methods such
● the ability for servers to provide executable code to the as GET, POST, PUT, DELETE, OPTIONS, and,
client hopefully, PATCH.
 REST: Pros vs Cons and Use cases
REST pros

REST cons: Decoupled client and server. Decoupling the client and the server as
much as possible, REST allows for a better abstraction than RPC. A
No single REST structure. There’s no exact right way to build a REST API. system with abstraction levels is able to encapsulate its details to better
How to model resources and which resources to model will depend on each identify and sustain its properties. This makes a REST API flexible enough
scenario. This makes REST simple in theory, but difficult in practice. to evolve over time while remaining a stable system.

Big payloads. REST returns a lot of rich metadata so that the client can Discoverability. Communication between the client and server describes
understand everything necessary about the state of the application just from everything so that no external documentation is required to understand
its responses. And this chattiness is no big deal for a big network pipe with how to interact with the REST API.
lots of bandwidth capacity. But that’s not always the case.
Cache-friendly. Reusing a lot of HTTP tools, REST is the only style that
Over- and under-fetching problems. Containing either too much data or allows caching data on the HTTP level. In contrast, caching
not enough of it, REST responses often create the need for another request. implementation on any other API will require configuring an additional
cache module.

Multiple formats support. The ability to support multiple formats for

storing and exchanging data is one of the reasons REST is currently a
prevailing choice for building public APIs.
REST use cases

Management APIs. APIs focused on managing objects in a system and intended for many consumers are the most common API type. REST helps
such APIs to have strong discoverability, good documentation, and it fits this object model well.

Simple resource-driven apps. REST is a valuable approach for connecting resource-driven apps that don’t need flexibility in queries.
Remote Procedure Call (RPC): invoking a function on
another system
A Remote Procedure Call is a specification that allows for remote
execution of a function in a different context. RPC extends the notion of
local procedure calling but puts it in the context of an HTTP API.

gRPC is the latest RPC version developed by Google in 2015. With

pluggable support for load balancing, tracing, health checking, and
authentication, gRPC is well-suited for connecting microservices.

How RPC works

A client invokes a remote procedure, serializes the parameters and
additional information into a message, and sends the message to a
server. On receiving the message, the server deserializes its
content, executes the requested operation, and sends a result back
to the client. The server stub and client stub take care of the
serialization and deserialization of the parameters.
 RPC: Pros vs Cons
RPC Cons
RPC Pros Tight coupling to the underlying system.
Straightforward and simple interaction. RPC’s tight coupling to the underlying system doesn’t allow for an
RPC uses GET to fetch information and POST for abstraction layer between the functions in the system and the
everything else. The mechanics of the interaction between external API. This raises security issues as it’s quite easy to leak
a server and a client come down to calling an endpoint and implementation details about the underlying system into the API.
getting a response. An RPC’s tight coupling makes scalability requirements and
loosely coupled teams hard to achieve. So, the client either
Easy-to-add functions. worries about any possible side effects of calling a particular
If we get a new requirement for our API, we can easily add endpoint or tries figuring out what endpoint to call because it
another endpoint executing this requirement: doesn’t understand how the server is naming its functions.
1) Write a new function and throw it behind an endpoint
and Low discoverability. In RPC there’s no way to introspect the API
2) now a client can hit this endpoint and get the info or send a request and start understanding what function to call
meeting the set requirement. based on its requests.
High performance.
Lightweight payloads go easy on the network providing Function explosion. It’s so easy to create new functions. So,
high performance, which is important for shared servers instead of editing the existing ones, we create new ones ending
and for parallel computations executing on networks of up with a huge list of overlapping functions that are hard to
workstations. RPC is able to optimize the network layer understand.
and make it very efficient with sending tons of messages
per day between different services.
REST vs RPC vs Others
In fact, some services are RESTful only to a degree. They
have RPC style at the core, break down larger services into
resources, and use HTTP infrastructure efficiently.

There really may be a gray zone between REST and RPC,

when a service implements some features of REST and
some of RPC. REST is based on the resource or noun
instead of action or verb-based.
Protocol development
Hardest problem in protocol design is precisely that we must try to expect
the unexpected.

Every protocol should be considered to be incorrect until the opposite is

proven
Protocol Development
Assume we have two computers, A and B. A is connected to a file store device d, and B connects to a
printer p. We want to send a text file from the file store at A to the printer at B

The two machines must have reached prior agreement on the meaning of control information and on
the procedures used to start, suspend, resume, and conclude transmissions. In addition, if
transmission errors are possible, control information must be exchanged to guard the transfer of the
data.

All rules, formats, and procedures that have been agreed upon between A and B are collectively called
a protocol:The protocol, then, can contain agreements on the methods used for:
● Initiation and termination of data exchanges
● Synchronization of senders and receivers
● Detection and correction of transmission errors
● Formatting and encoding of data
Protocols representation
An important part in designing protocols is to
specify in a very detailed, non-ambiguous way,
how it works. Natural language is ambiguous, and
in order to alleviate this problem, special domain
language with very strict definitions, and tons of
documentation are usually needed.

An alternative, would be to use something a priori

formal, an automaton, or state machine.

UML protocol state machine diagrams overview,

show usage protocol or a lifecycle of a classifier
Protocols representation
Sequence diagrams normally will show you
only one, “happy” flow of messages between
clients and servers without becoming cluttered.
This sometimes results in a number of
sequence diagrams to formally describe the
protocol.

Networking Protocol Sequence Diagrams ->

Protocol Definition
Protocol definition can be compared to a language
definition: it contains a vocabulary and a syntax
definition (i.e., the protocol format); the procedure
rules collectively define a grammar; and the service
specification defines the semantics of the language

There are some special requirements we have to

impose on this language. Like any computer
language the protocol language must be
unambiguous. Unlike most programming languages,
however, the protocol language specifies the
behavior of concurrently executing processes.
This concurrency creates a new class of subtle
problems. We have to deal with, for example, timing,
race conditions, and possible deadlocks.
Protocol Specifications
A protocol specification consists of five distinct parts. To be complete, each
specification should include explicitly:

1. The service to be provided by the protocol

2. The assumptions about the environment in which the protocol is executed
3. The vocabulary of messages used to implement the protocol
4. The encoding (format) of each message in the vocabulary
5. The procedure rules guarding the consistency of message exchanges
Protocol Service Specification
Informal example of the definition of the five protocol elements, and
the types of errors that can linger in a design. Following that, we
consider each of the five main protocol elements in more detail.

SERVICE SPECIFICATION
The purpose of the protocol is to transfer text files as sequences of
characters across a telephone line while protecting against
transmission errors, assuming that all transmission errors can in fact
be detected. The protocol is defined for full-duplex file transfer, that
is, it should allow for transfers in two directions simultaneously.
Positive and negative acknowledgments for traffic from A to B are
sent on the channel from B to A, and vice versa. Every message
contains two parts: a message part, and a control part that applies
to traffic on the reverse channel.
Protocol Assumption About The Environment
The ‘‘environment’’ in which the protocol is to be executed consists minimally of
two users of the file transfer service and a transmission channel.
The users can be assumed to simply submit a request for file transfer and await its
completion.
The transmission channel is assumed to cause arbitrary message distortions, but
not to lose, duplicate, insert, or reorder messages.
Protocol Vocabulary
The protocol vocabulary defines three distinct types of messages:

● ack for a message combined with a positive acknowledgment,

● nak for a message combined with a negative acknowledgment
● err for a message with a transmission error.

The vocabulary can be succinctly expressed as a set: V = { ack, err, nak }.

Each message type can further be refined into a class of lower-level messages,
consisting for instance of one sub-type for each character code to be transmitted.
Protocol Message Format
Each message consists of a control field identifying the message type and a data field with
the character code.
For the example we assume that the data and control fields are of a fixed size.

The general form of each message can now be represented symbolically as a simple
structure of two fields:
{ control_tag, data }
which in a C-like specification may be specified in more detail as follows:
enum control { ack, nak, err };
struct message {
enum control tag;
unsigned char data;

};
Protocol Procedure Rules
The procedure rules for the protocol were
informally described as follows:

1. If the previous reception was error-free,

the next message on the reverse channel
will carry a positive acknowledgment; if
the reception was in error it will carry a
negative acknowledgment.

2. If the previous reception carried a

negative acknowledgment, or the previous
reception was in error, retransmit the old
message; otherwise fetch a new message
for transmission.
Protocol Procedure Rules
First, process A initiates the transfer by sending a deliberate
error message to B. Assume that A attempts to transmit the
characters a to z, and that B responds by transmitting the
characters in the reverse order, from z to a.
Consider then the sequence of events shown in the time
sequence diagram. The two solid lines in the figure track the
executions of the two processes. The dotted lines show
successful message transfers. The dashed lines show
message transfers that are distorted by the channel. Two
messages are distorted in this manner: a positive
acknowledgment from A to B and a negative
acknowledgment from B to A.
At the end of the sequence, when A receives the last
message from B, it cannot tell whether the message is new
or an old duplicate. The nak message that contained this
information was corrupted. In the example sequence, A
erroneously accepts the message.
Protocols Error detection
Errors in transmission happen for whatever reasons. Most common ones are rearranged bits, bit flips,
duplicate bits, deleted/lost bits, and sometimes even inserted bits. All it depends on the network
characteristics.
For example, the probability of an error in computer system circuits is 1e-15, in case of optical fiber
cables - 1e-9, coaxial ones - 1e-6, and in case of telephone links - between 1e-4 and 1e-5. The
difference is huge indeed.
Just so you would understand better, “At a rate of 9600 bits per second, it [error probability of 1e-15]
would cause one single bit error every 3303 years of continuous operation. At the same data rate, a bit
error rate of 1e−4 causes a bit error, on average, once a second.”
Errors are usually of 2 types:
- Linear distortion of the original data, for instance, as caused by bandwidth limitations of the raw
data channel
- Non-linear distortion that is caused by echoes, cross-talk, white noise, and impulse noise
Also, depending on the characteristics of a given system, it might be more probable to get single bit
errors, that do not “cascade” (like RAM memory, or CPU caches) or, more likely in practice, burst errors,
that is, if an error occurs for a given bit/byte/sequence, the probability of the next sequence to be
distorted is much higher than the baseline. This understanding is important when choosing how the
system will cope with errors.
Protocols Error detection: Parity Checking
If the probability of multiple bit errors per message is
sufficiently low, all the error control needed on a binary
symmetric channel is a parity check code. To every
message we add a single bit that makes the modulo-2
sum of the bits in that message equal to one. If any
single bit, including the check bit, is distorted by the
channel the parity at the receiver comes out wrong and
the transmission error can be detected.

The parity bit ensures that the total number of 1-bits in

the string is even or odd. Accordingly, there are two
variants of parity bits: even parity bit and odd parity bit.
In the case of even parity, for a given set of bits, the
occurrences of bits whose value is 1 are counted.
 Protocols Error Detection: Hamming Code
An example of a code that realizes this protection is a code
developed by R. Hamming.

The check bits have been placed in the code word in such a
way that the sum of the bit positions they occupy points at the
erroneous bit for any single bit error. To catch a single bit error
the check bits are used as parity bits.

When a bit position is written as a sum of powers of two, for

example, (1 + 2 + 4), it also points at the check bits that cover
it. Data bit 7 = ( 1 + 2 + 4 ), for instance, is counted in the
three check bits at positions 1, 2, and 4. A single bit error that
changes the seventh data bit changes the parity of precisely
these three checks. The receiver can therefore indeed
determine which bit is in error by summing the bit positions of
all check bits that flagged a parity error. An error that changes,
for instance, the second bit only affects that single bit and can
also be corrected

Hamming Code | Error detection

Protocols Error Correction: Hamming Code
The ASCII character code for the letter D is 1000100.
Image shows how the data and parity bits are placed
in a Hamming code. If a transmission error changes
bit position 7 from a 0 into a 1 the code arrives as the
ASCII code for an L 1001100. But, the first three
parity bits transmitted now differ from the values the
receiver can calculate and reveal the faulty seventh
bit.
It is of course not really relevant to the code as such
in what order the code bits are placed in a code
word. By rearranging the bits, for instance, every
binary Hamming code can be changed into a
systematic code or into a cyclic code.

Hamming Code | Error Correction

Protocols useful materials
DESIGN AND VALIDATION COMPUTER
PROTOCOLS