Dr. Arif Butt

Sr.Topic DescriptionLecture ResourceSessional Instruments
1

 

Module-1 (Overview of Course):

A discussion on course information and protocols. Evolution of technology and data generation sources. Structured, semi-structured and un-structured data. What is Data Science? Factors making data science ubiquitous. Applications of data science in the domains of social media, banking, e-commerce, web-based search engines, travelling, health care, automation, credit and insurance.

 

How to do data science? Who is a data scientist? Skill sets of a data scientist. Programming languages, tools and techniques involved in doing data science. 3 Vs of Big Data and tools used to handle Volume, Variety, and Velocity. Python libraries for data science tasks. Why data science is so complicated? Six phases of data science life cycle: Business Problem, Data Acquisition, Data Processing, EDA and Visualization, ML Model Creation-Training-Evaluation, Deployment and Monitoring. Industry job roles and salary trends in data science domain.

Lec-1.1 1.2VL-1.1
Course GitHub Repository
VL-1.2
2

VERSION CONTROL SYSTEMS:
Overview and types of versioning control systems: Local, Centralized and Distributed Data Models. Downloading, installing, and configuring git on local machine. Overview of git three tree architecture and its basic work flow. Branching, merging, and handling merge conflicts. Web Portals and Cloud hosting services for git (GitHub, Bitbucket, GitLab). Creating remote repository on GitHub. Cloning a remote repository on local machine. Pushing a local repository on remote repository. Fetch vs Pull vs Fork

Lec-1.3
RPaper-01
RPaper-02
RPaper-03
VL-1.3
3

 

Module-2 (Basics of Python Programming):

PYTHON ENVIRONMENT:
History of Python, its key features, important libraries and frameworks. Applications of Python. World top organizations using Python. Downloading and installing latest version of Python. Code Editors (Notepad , Atom, Sublime), Desktop IDEs (Spyder, Pycharm), Browser based IDEs (Jupyter Notebook/Lab), and Cloud based IDEs (Google Colab, Deepnote, CoCalc). Introduction to IPython, Installing Anaconda navigator and Jupyter Notebook, Jupyter Notebook architecture, converting notebook formats and sharing notebooks. A hello world of Python Programming in Jupyter Notebook. Handling Numeric data types in Python. Arithmetic, assignment, comparison, logical, bitwise, identity and membership operators

 


STRING:
Defining strings in Python, Accessing characters of a string in Python, Strings are immutable, Slicing strings, String concatenation, Creating large strings, String Methods: lower(), upper(), strip(), startswith(), split(), join(), find(), replace(), format(), String Membership test

Lec 2.1
Lec 2.2
Lec 2.3
Hello Jupyter
Markdown
Variables
Numbers
Operators
Python Strings
VL-2.1
VL-2.2
VL-2.3
VL-2.4
4

LIST:
What are Python Lists and how to create them? Different ways to access list elements, Slicing Lists, List concatenation and repetition, Modifying/Adding elements to a List using append(), extend(), and insert() methods, Removing elements from a list using pop() and remove() methods, Converting string object to list and vice-versa, The sort() and reverse() methods, Use of in Operator on Lists, Comparing objects and values, Simple Assignment (aliasing) vs Shallow Copy vs Deep Copy, Pickling vs Un-pickling in Python

TUPLE:
What are tuples and how to create them? Accessing elements of tuples? Tuple concatenation and repetition, Slicing a tuple, Converting string object to tuple and vice-versa, Misc tuple methods and Concepts

SET:
What are sets and how to create them? Accessing elements of sets? Adding elements to a sets using add, and update methods, Removing elements from a set using pop, remove and discard methods. Converting string object to set and vice-versa, Misc set methods. Union, Intersection, Difference, Symmetric Difference of sets, Subsets, Supersets and Disjoint sets

DICTIONARY:
What are dictionaries and how to create them?, Proof of concepts, Accessing elements of a dictionary, Adding/Modifying elements in a dictionary, Removing elements from a dictionary, Dictionary, tuple and list conversions, Misc Dictionary methods

Python Lists
Tuples
Sets
Dictionary
VL-2.5
VL-2.6
VL-2.7
VL-2.8
5

SELECTION STRUCTURE:
Python Indentation, Branching with if, else, and elif, Ternary operator, Nested if-else statement, if-elif ladder, Statements and Expressions, The pass statement, Python3 input() function

REPETITION STRUCTURE:
While loop (Indefinite Iteration), Break and continue statements, While loop with else statement, For loop, Iterables and Iterators, Range and enumerate() functions, Nested for loop

FUNCTIONS:
What are functions in Python, User Defined Functions in Python, Docstring inside a function, Passing arguments and Returning value from a function, Pass by value vs Pass by reference, Arguments of a Python function, Positional/Required arguments, Default arguments, Keyword arguments, Variable length arguments, Arbitrary keyword arguments, Nested functions, Understanding Scope in Python, Local scope, Enclosing scope, Global scope, Built-in Scope, Anonymous Functions or Lambda Functions, Basic Examples of lambda functions, Using lambda function as argument to other functions, Using Lambda Function with built-in sorted(), map(), filter(), and reduce() functions

If-Else
Loops
Functions 1
VL-2.9
VL-2.10
VL-2.11
6

EXCEPTIONS:
What are Exceptions? Raising an Exception, Handling exceptions using try-except block, Types of exceptions in Python, Multiple except blocks, Try-except with else clause, Finally keyword in Python, User defined exceptions

BUILT IN MODULES AND PACKAGES:
What are Python packages, modules, and libraries. Python Math, Random, Time, DateTime, Calendar, OS, Threading and URLLIB Modules

CREATING MODULES AND PACKAGES:
How to create a Python Module? How to use a Module in Python? How Python locates a Module? Reloading a Python Module, What are Python Packages? How to create a Python Package? How to import modules from a Package?

Functions 2
Exceptions
Modules 1
Modules 2
VL-2.12
VL-2.13
VL-2.14
VL-2.15
7

FILE HANDLING:
What are Files, and Why we need them? Opening a file, Reading contents of a file using read(), readline() and readlines(). Iterating a file object, Changing file offset using seek(), Writing and appending data in a file, Close a file, Delete a file, Rename a file, Get file attributes using stat(), Checking file types, Directory handling, Create a directory, Delete a directory, Rename a directory, Changing a process cwd, Handling Image Files in Python, Opening an image file using open() and read(), Reading and displaying image files using library functions, Copying image file, Cropping image file, Transposing image file, Saving image file

File HandlingVL-2.16
8REGULAR EXPRESSION:

Python re module, match() and search() function, matching vs searching, Regular expression modifiers, patterns, and special sequences. The match(), findall(), search(), split() and sub() functions

RE-1
RE-2
VL-2.17
VL-2.18
9

 

Module 3 (Python for Data Scientist):

NUMPY:
Introduction to Numpy Library. Array Creation using np.array() constructor. Miscellaneous Array Creation Functions: np.zeros(), np.ones(), np.empty(), np.full(), np.eye(), np.fromstring(), np.arange(), np.linspace(), np.random.rand(), np.random.randint(), np.zeros_like()
A Comparison between Python Lists, Python Arrays and NumPy Arrays. Memory Consumption of Python List and Numpy Array. Operation cost on Python List and Numpy Array

 

NumPy-01
NumPy-02
VL-3.1
VL-3.2
10

NUMPY:
NumPy universal functions (unary and binary), Scalar Math, Arithmetic Operations, Aggregate Functions, Comparison Operations. Searching NumPy Arrays.
Slicing NumPy Arrays. Boolean and Fancy Array Indexing.
Broadcasting NumPy Arrays. Reshaping NumPy Arrays using the shape attribute, np.reshape(), np.resize(), ndarray.transpose(), np.swapaxes(), np.flatten() methods. Sorting Arrays using np.sort(). Iterating NumPy Arrays

NumPy-03
NumPy-04
NumPy-05
VL-3.3
VL-3.4
VL-3.5
11

NUMPY:
Updating existing values of NumPy array elements. Append new elements to a NumPy array using np.append(). Insert new elements in a NumPy array using np.insert(). Delete elements of a NumPy array using np.delete()
Alias vs Shallow Copy vs Deep Copy
Array concatenation, stacking and splitting.
Reading and Writing Numeric data from text/csv Files to NumPy Arrays

NumPy-06
NumPy-07
NumPy-08
VL-3.6
VL-3.7
VL-3.8
12

PANDAS:
Overview of Pandas Library and its Data Structures. Install Pandas Library. Read Datasets into Pandas Dataframe. Python Dictionaries vs Pandas Dataframes. Anatomy of a Dataframe
Creating a Series
– From Python List
– From NumPy Arrays
– From Python Dictionary
– From a scalar value
Attributes of a Pandas Series. Understanding Index in a Series and its usage: (Identification, Selection, Subsetting, Alignment)

Panda-01
Panda-02
Panda-03
VL-3.9
VL-3.10
VL-3.11
13

PANDAS:
Anatomy of a Dataframe, Creating Dataframe from an empty dataframe, from Two-Dimensional NumPy Array, from Dictionary of Python Lists, from Dictionary of Panda Series. Attributes of a Dataframe
Reading data from a CSV/TSV File Reading a CSV file from a Remote System. Writing Contents of Dataframe to a CSV File. Reading data from an EXCEL File. Writing Contents of Dataframe to an EXCEL File. Reading data from a JSON File. Writing Contents of Dataframe to a JSON File

Understanding Indices of a Dataframe, Selecting Row(s) and Column(s) of a Dataframe using `df[]`, Selecting Rows and Columns using `iloc` Method, Selecting Rows and Columns using `loc` Method, Conditional Selection, Selecting columns of a specific data type

Modifying Column labels of Dataframe. Modifying Row indices of Dataframe. Modifying Row(s) Data (Records) of a Dataframe. Modifying multiple Rows using `map()`, `df.remove()`, `df.apply()`, `df.applymap()` methods

Panda-04
Panda-05
Panda-06
VL-3.12
VL-3.13
VL-3.14
14

PANDAS:
Reading a Sample Dataframe, Adding a New Column or Deleting an Existing Column from a Dataframe, Adding a New Row or Deleting an Existing Row(s) from a Dataframe, Adding a New Column with Conditional Values, Deleting Row(s) Based on Specific Condition, Delete a Column Based on Specific Condition, Change Datatype of a Pandas Series, Sorting dataframes using `df.sort_values()`, `df.sort_index()` methods.

Have an insight about the Dataset, Identify Column(s) containing Null/Missing values using df.isna() method, Handle/Impute the Null/Missing Values under the Column using `df.loc`, Handle Missing values under a Numeric/Categorical Column using fillna() method, Handle Repeating Values (for same information), Create a new Column by Modifying an Existing Column, Delete Rows containing NaN values using df.dropna() method, Convert Categorical Variables into Numerical.

Overview of Aggregation Functions and the `agg()` method, Computing the Minimum Temperature of different Cities using hard way, groupby methods. Practice GroupBy on Stack Overflow Survey Dataset.

Panda-07
Panda-08
Panda-09
VL-3.15
VL-3.16
VL-3.17
15

PANDAS:
Merging DataFrames using pd.merge() method, Perform Inner Join (which is default), Perform Outer/Full Outer Join, Perform Left Outer Join, Perform Right Outer Join, Additional Parameters to pd.merge() Method, Concatenation using pd.concat() method, Appending Dataframes using df.append() method.

Reshape Data Using `pivot()` , `pivot_table()` , `melt()` and `crosstab()` methods.

Recap of Python’s Built-in Time and Datetime Modules, Overview of Pandas Time Series Data Structures, Converting Strings to Pandas DateTime64 type, Convert a Scalar String to DateTime, Convert Pandas Series to DateTime, Handling Issues of DateTime Formats, Convert a Single Integer to Pandas DateTime, Practicing with a Simple Dataset, UFO Dataset, and Crypto-Currency Dataset, Concept of Up Sampling and Down Sampling, Bonus Task.

Panda-10
Panda-11
Panda-12
VL-3.18
VL-3.19
VL-3.20
16

EDA AND VISUALIZATION:
Overview of Data Visualization, Choosing an appropriate Chart for the problem at hand, Anatomy of a Figure, Download and Install Matplotlib, Programming with Matplotlib, How to draw a Line Chart, Enhance the Graph Step by Step and Saving it on Disk. Understanding Sub-Plots.

Understanding Bar-plot, Visualizing Bar-Plot for Stack Overflow Dataset, Understanding Scatter-plot, Visualizing Scatter Plot for Houses Dataset.

Understanding Pie Chart, Visualizing Pie Chart for Stack Overflow Survey Dataset, Understanding Histogram, Visualizing Histogram for Stack Overflow Survey Dataset, Visualizing Histogram of an Image.

Visualization-I
Visualization-II
Visualization-III
VL-3.21
VL-3.22
VL-3.23
17

EDA AND VISUALIZATION:
Recap of previous session, Understanding/Visualizing Box Plot, Violin Plot, Heatmap
Creating Multiple Sub-Plots within a Figure Object.

Overview of Seaborn Library, Downloading and Importing Seaborn Library, Built-in Datasets of Seaborn Library, Car_Crashes, Flights, Tips, Iris, Titanic Datasets. Three Figure Level Methods of Seaborn. Using sns.relplot() method to draw line and scatter plots. Using sns.catplot() method to draw bar, count, box, violin, strip, and swarm plots. Using sns.displot() method to draw hist, kde and ecdf plots.

Visualization-IV
Visualization-V
VL-3.24
VL-3.25
18

 

Module-4 (Mathematics for Data Science):

DESCRIPTIVE STATISTICS:
Overview of Overview of Section 4, Data and its Types, Population vs Sample.
Overview of Statistics and its Types Descriptive Statistics vs Inferential Statistics.
Measure of Center (Mean, Median and Mode). Measure of Spread (Range, IQR, Variance, and Standard Deviation). Measure of Asymmetry (Skewness and Kurtosis).
Analysis Strategy, Variance vs Covariance vs Correlation, Covariance and Covariance Matrix, Correlation and Correlation Matrix, Python Implementation of Covariance and Correlation.
Standardizing/Normalizing a Dataset, Overview of Titanic, Iris, Tips Datasets. Practicing Bivariate Data Analysis.

 

Descriptive-StatisticsVL-4.1
19

INFERENTIAL STATISTICS:
Overview of Probability, Joint Probability, Combinatorics, Practical Implementation in Python.
The Law of Large Numbers, Conditional Probability, Bayes’ Theorem, Applications of Bayes’ Theorem, Naive Bayes’ Classifier, How Probability Relates with Statistics.
Probability Distributions for Discrete Random Variables, Probability Distributions for Continuous Random Variables, Normal Distribution, Standard Normal or Z-Distribution, Practical Implementation of Distributions in Python.
Z-Score vs P-Value, Central Limit Theorem.

How to formulate a Hypothesis? Types of Hypothesis Tests. Related Terminologies (Rejection region, significance level, test scores, p-value, and error types).
Student’s Single Sample T-Test Student’s Two Independent Samples T-Test, Student’s Two Paired Samples T-Test.

Variance and Standard Deviation, Covariance and Covariance Matrix, Correlation and Correlation Matrix, Regression Analysis, Linear Regression, Fitting a Line (Gradient Descent), Fitting a Line (Linear Least Squares).

Inferential-StatisticsVL-4.2
20

LINEAR ALGEBRA I:
Overview of Vectors, Mathematical and Graphical Representation of Vectors in R2, Mathematical and Graphical Representation of Vectors in R3, Hands on Implementation in Python, Magnitude of a Vector (Vector Norms), Direction of a Vector, Components of a Vector in R2, Components of a Vector in R3, Vector Addition, Multiplying a Vector with Scalar value, Basis and Unit Vectors, Linear combination and Span of Vectors, Vector Dot Product, Vector Cross Product, Orthogonal and Orthonormal Vectors.

Row Vector vs Column Vector, Zero Matrix, Ones Matrix, Random Integer Matrix, Square Matrix, Symmetric Matrix, Triangular Matrix, Diagonal Matrix, Identity Matrix, Scalar Matrix, Orthogonal Matrix, Matrix Addition, Matrix-Scalar Multiplication, Matrix Multiplication (Hadamard Product), Matrix Multiplication (Dot Product), Matrix-Vector Multiplication, Frobenius Norms, Transpose of a Matrix, Determinant of a Matrix, Inverse of a Matrix, Trace of a Matrix, Rank of a Matrix.

An overview of Linear Equations, What is a system of Linear Equations? How to solve a system of Linear Equations? Substitution, Elimination and Graphing strategy, Consistent vs Inconsistent System of Linear Equations, Plotting a Linear Equation with Three Variables, Solving set of Three Linear Equations with Three variables, Solving System of Linear Equations using Matrix Algebra, Writing a system of Linear Equations in Matrix form, Solving system of Linear Equations using Gaussian Elimination Method, Solving system of Linear Equations using Gauss Jordan Method, Solving system of Linear Equations using Cramer’s Rule, Solving system of Linear Equations using Matrix Inverse Method, Limitations of Matrix Inversion Method, Categories of System of Linear Equations, Overdetermined systems, Underdetermined systems, Solving Inconsistent Overdetermined System of Linear Equations using Least Squares Method, Modeling Linear Equations in Machine Learning with `2` variables, Modeling Linear Equations in Machine Learning with `m` variables, Simple Linear Regression using Least Squares Method, Multiple Linear Regression using Ordinary Least Squares (OLS) Method.

Linear-Algebra-IVL-4.3
21

LINEAR ALGEBRA II:
What do you mean by applying a Matrix to a Vector? Algebra Behind Linear Transformations in 2-D Space, Applying an Identity Matrix, Applying a Scaling Matrix, Applying a Reflection Matrix, Applying a Shear Matrix, Applying a Rotation Matrix, Applying a Translation Matrix, Apply Multiple Transformations Simultaneously. Scaling, Translation and Rotation in Three Dimensional Space.

Eigenvalues and Eigenvectors (An abstract view), Calculating Eigenvalues and Eigenvectors of a Matrix using Paper-Pencil, Calculating Eigenvalues and Eigenvectors of a Matrix in Python, Relationship between Determinant and Eigenvalues of a Matrix, Relationship between Trace and Eigenvalues of a Matrix, Not all Transformation Matrices have Real Eigenvalues, Eigenvectors and Eigenvalues of a Diagonal Transformation Matrix, Eigendecomposition, Eigendecomposition for Symmetric Matrices, What is Singular Value Decomposition? Calculating SVD using NumPy, Calculating SVD using SciPy, Calculating SVD using Scikit-learn.

Image Compression, What is Principal Component Analysis?
Example 1: Dimensionality Reduction of Housing Dataset using NumPy,
Example 2: Dimensionality Reduction of Digits Dataset using Scikit-learn,
Example 3: Dimensionality Reduction of Iris Dataset using Scikit-learn (Data Visualization), The Moore-Penrose Pseudoinverse.

Linear-Algebra-IIVL-4.4
22

CALCULUS FOR MACHINE LEARNING:

CalculusVL-4.5
23

 

Module-5 (Web Scrapping):

DATA ACQUISITION:
Overview of Web Scraping Using APIs to Acquire Data from Websites, What is Web Scraping? How Web Scraping Works? BeautifulSoup vs Selenium vs Scrapy, Is Web Scraping Legal? A bit of Motivation, Overview of a Web Application Architecture, The Anatomy of a Web Page, The Document Object Model (DOM), Google Chrome Developer Tools, The Python urllib Package, The requests Library.

 

Overview of BeautifulSoup and Requests Library, Download and Install BeautifulSoup, Playing with BeautifulSoup and Requests, Reviewing the Books Scraping Website, Fetching HTML contents using Requests Library, Creating Soup object and playing with it using the soup.find() and soup.find_all() methods. Example 1: Scraping data of a single page, Saving Data into a CSV file, Scraping Data from Multiple web pages. Example 2: Scraping data from a website that employs pagination, Limitations of Requests and BeautifulSoup.

Web-Scrapping-1
Web-Scrapping-2
VL-5.1
VL-5.2
24

DATA ACQUISITION:
Overview of Web Scraping using Selenium, Selenium (Why, What and How), Architecture of Selenium, Download and Install Selenium and its WebDriver Component, A step-by-step Hello World with Selenium, Create an Instance of a Web Driver, load a web page, play and quit.
Example 1: Scraping JS Driven Website.
Example 2: Scraping Website behind a login. Selenium find_element() and find_elements() methods. Understanding Locators in Selenium. Consolidated Script for Example 2.
Example 3: Scraping Infinite Scrolling Website.
Example 4: Scraping Pagination Website.
Example 5: Scraping Website containing pop-ups. Bonus: Sending email with attachment from Python.

Best Practices for Web Scraping.
Example 1: Searching and Downloading Images
Example 2: Scrape YouTube video comments
Example 3: Scraping jobs from a Job Website
Example 4: Scraping Tweets of a celebrity
Example 5: Scraping News Paper articles.

Web-Scrapping-3
Web-Scrapping-4
VL-5.3
VL-5.4
25

 

Module-6 (Overview of Machine Learning):

Under Process

 

ML-1
ML-2
VL-6.1
VL-6.2
26

Under Process

ML-3
ML-4
VL-6.3
VL-6.4
27

Under Process

ML-5
ML-6
VL-6.5
VL-6.6
28

Under Process

ML-7
ML-8
VL-6.7
VL-6.8
29

 

Module-7 (Overview of NLP):

Overview of NLP, Challenges of NLP (Language Ambiguities), Real World Applications of NLP, Approaches used to solve NLP use cases, Heuristic Method Approach, Machine Learning Approach, Deep Learning Based Approach, NLP Pipeline.

 

Overview-NLPVL-7.1
30

PRE-PROCESSING:
Removing digits and words containing digits, Removing newline characters and extra spaces, Removing HTML tags, Removing URLs, Removing punctuations.

Case folding, Expand contractions, Chat word treatment, Handle emojis, Spelling correction, What is Tokenization? Tokenization using string.split() Method, Tokenization using re.split() Method, Tokenization using NLTK library, Tokenization using spaCy library, Creating N-grams, What are Stopwords? Removing Stopwords using NLTK library, Removing Stopwords using spaCy library, Text Preprocessing on IMDB Dataset.

Stemming using NLTK’s PorterStemmer, Lemmatization using NLTK’s WordNetLemmatizer, Parts of Speech Tagging using spaCy library, Named Entity Recognition using spaCy library, Coreference Resolution.

Text-Preprocessing-I
Text-Preprocessing-II
VL-7.2
VL-7.3
31

FEATURE ENGINEERING:
Overview of Feature Engineering, Label Encoding, One-Hot encoding, Bag of words, Bag of n-grams, TF-IDF, Overview of Word Embeddings, Word2Vec using spaCy Large English Model, Word2Vec using Gensim and Google News Model.

Feature-EngineeringVL-7.4
32

SPAM DETECTION:
Overview of Text Classification, From Bayes’ Theorem to Naïve Bayes’ Classifier, Marginal Probability, Independent Events and Joint Probability, Dependent Events and Conditional Probability, From Conditional Probability to Bayes’ Theorem, Naïve Bayes’ Classifier.
Example 1: Naïve Bayes’ Classifier (Single Input Feature)
Example 2: Naïve Bayes’ Classifier (Multiple Input Features)
Example 3: Naïve Bayes’ Classifier (Text Data)
Data Acquisition for Email Spam Classification, Adding new features in the dataset, Exploratory Data Analysis, Text Pre-Processing, Text Vectorization using Bag of Words (BoW), Model Building (Feed Training Data to Machine Learning Model), Evaluation of Machine Learning Model, Confusion Matrix, Accuracy, Accuracy Paradox, Precision, Recall, Limitations of Precision and Recall, F-1 Score, Model Deployment, Creating a PipeLine Object for Predicting New Incoming Messages.

Spam-DetectionVL-7.5
 
Sr.Topic DescriptionLecture ResourceSessional Instruments
1

Module-1 (Preparing your toolbox):
Overview of Operating System. Review of Computer Hardware, Learning environment, Accessing Operating System services, Discussion on Course Matrix. How system calls work (Behind the curtain). Making a system call with and without wrapper. Making a system call from within Assembly program. (Tools used are are gcc, gdb, readelf, objdump, nasm)

Lec1.0
SPVL-01
SPVL-02
RPaper-01
RPaper-02
 
2

Linking and loading a process (Behind the curtain). Load time and run time dynamic linking. Structure of ELF file format. Merging re-locatable object files into executable. What is relocation. Symbol (global, external, local) resolution. Strong and weak symbols. Linker symbol rules for multiple strong and weak symbols. Creating and using your own static libraries. Creating and using your own dynamic libraries or shared objects. (Tools used are nm, ar, ranlib)

Lec1.1
SPVL-03
 
3

Introduction to make utility, how make works, Makefile rules and targets. Using your own and built-in macros in a Makefile. Writing and calling multiple make files for a program. Comparison between binary and open source software packages. Downloading and installing open source software. Packaging your own software using GNU autotools (autoconf and automake). Packaging your own software using cmake utility. Writing your own man pages. (Tools used are make, autoconf, automake, cmake, cpack)

Self Reading/Performing Task (Versioning Systems git)

 

Lec1.2
Lec1.3
SPVL-04
SPVL-05
SPVL-06
SPVL-07
 
4

How a C program starts and terminates. Normal vs abnormal termination. Registering exit handlers using atexit() and on_exit() functions. Querying process resource limits and changing them on the shell. Getting and setting process resource limits from within a C program using getrlimit() and setrlimit() functions. Memory layout of a C program. Use of command line arguments and environment variables in C programs. A discussion on layout of process stack and how it grows and shrinks. Stack buffer overflow problem. Doing a non-local goto using longjmp() and setjmp()Allocating using and freeing memory on heap. Layout of heap and heap allocators. Role of system calls brk() and sbrk() in management of heap. Common programming errors related to heap management. Tools and libraries for malloc debugging like splint, electric-fence, and valgrind

Lec1.4
Lec1.5
Lec1.6
SPVL-08
SPVL-09
SPVL-10
 
5

Module-2 (File System Architecture):

Hard disk partitioning, formatting and mounting. Internal structure of UNIX file system (Disk, partitions, boot block, super block, inode block). Connection to an open file via PPFDT. Universal I/O model (open-read-write-close paradigm). Kernel Buffering of file I/O and Buffering in the stdio library. Misc important file management system calls. Repositioning current file offset using lseek(). Creating soft link to a file, changing permissions, and ownership of a file.

Lec2.1
SPVL-12
RPaper-03
RPaper-04
 
6

Different techniques of I/O redirection. Use of fcntl() system call to duplicate a file descriptor, get/set file descriptor flags in PPFDT, get/set file status flags in SWFT, and achieving locks on files.

SPVL-13 
7

Directory management in C Programs. Working of UNIX ls utility. How it works? Can we design ls utility of our own? Design and code of UNIX ls utility.

Self Reading/Performing Task (Design and Code of UNIX more and who utility, Lec2.3 and Lec2.4)

 

Lec2.2
Lec2.3
Lec2.4
SPVL-11
SPVL-14
SPVL-15
 
8

Introduction to terminal devices. Device files and their attributes. Writing to someone else’s terminal. Attributes of a terminal driver (Input, output, control, and local processing). Modifying terminal attributes on the shell using stty command. Canonical and non-canonical modes of terminal driver. Programming the terminal driver using tcgetattr(), tcsetattr() and ioctl()

Self Reading/Performing Task (Design and Code of UNIX stty utility)

 

Lec2.5
SPVL-16
 
9

UNIX IO Models: Blocking IO model, Non-blocking IO model, Multiplexed IO model, Signal driven IO model, Asynchronous IO model.

Lec2.6 
10Module-3 (OS Structures, Kernel Compilation and LKM):

Introduction to operating system structures, Monolithic, DOS and Microkernel structures, SPIN approach to OS structure, Exokernel structure, L3 Microkernel structure.

Lec3.1
RPaper-5
RPaper-6
RPaper-7
 
11Overview of Booting Process and Kernel Initialization. Introduction to Compiling an Operating System Kernel from Source code, Preparing your work environment, Downloading the Kernel Source code, Configure Kernel, Compile/build the Kernel, Install the Kernel, Update the Boot loader, Verify the installation. Adding System call to the Kernel.Introduction to Linux Kernel Modules (LKM) and its usage, Writing, loading and unloading a hello world LKM of your own.Lec3.2
Lec3.3
Lec3.4
 
12Module-4 (Process Management and Scheduling):

Overview of the data structures maintained by Linux kernel to manage processes. Process Identifiers: getpid(), getppid(), getuid(),setuid(), getgid(),setgid(), geteuid(),seteuid(), getegid(), setegid(). Process creation using fork(), vfork(), clone(). Copy on Write Semantics. Orphan and Zombie process. Process trees, chains and fans.

Lec4.1
SPVL-17
RPaper-8
RPaper-9
 
13

Monitoring Child Processes by accessing termination status of the child process using wait(), waitpid(), waitid(), wait3(), wait4() and WIFEXITED(), WEXITSTATUS(), WIFSIGNALED() and WTERMSIG() macros
The exec() family of functions: execl(), execle(), execlp(), execv(),execve(), execvp() Process Groups, Process Sessions, and concept of Controlling Terminals. Impact of fork() and exec() on different process attributes. writing your our own system() function and use it to create a shell utility

Lec4.2
Lec4.3
SPVL18
SPVL19
 
14

Overview of daemon processes in Linux. Writing your own daemon process programatically. Introduction to systemd (replacement of SysV init daemon). Controlling daemons using systemctl utility. Writing long lived process and managing it using systemctl.

Design and Code your own daemon using systemd.

Lec4.4
SPVL-20
 
15

UNIX SYS-VR3 and SYS-VR4 scheduler. Linux O(1) and CFS scheduler. Cache affinities and multicore, Cache aware scheduling.
Accessing and changing scheduling parameters on the shell using schedtool and from within a C program using Linux API

Design and code of Shell utility (SPVL-22)

Lec4.5
Lec4.6
SPVL-21
SPVL-22
 
16

Different ways to achieve concurrency. Conventional Concurrent Servers and their Problems. Thread implementation models. Linux implementation of POSIX threads (LinuxThreads and NPTL). Thread creation and termination, joining threads, passing implicit data types to thread function, and passing structures to thread function. Returning and receiving data from a thread function. Creating Thread arrays on stack, and creating thread arrays on heap. Thread attributes (detachstate, stackaddr, stacksize, priority, inheritsched). Changing the default attributes of a thread. Threads and signals. Threads and fork(). Thread cancellation.

Lec4.7
SPVL-23
 
17

Module-5 (Inter Process Communication):

Ways to share information among UNIX processes. Taxonomy of UNIX Inter Process Communication (Communication, Synchronization, and Signals). Persistance of IPC objects. Overview of Signals, Standard and Real time Signals. Signal handling on Bash shell using kill and top utilities. Sending signals using kill(), raise(), abort(), pause(), alarm(), sleep(), usleep(), nanosleep() functions.
Ignoring and handling signals using signal() system call. Avoiding race conditions using signal mask and sigprocmask() system call. Limitations of signal() system call. Ignoring and handling signals using sigaction() system call. Scheduling future actions using interval timers getitimer(), setitimer()
Facts about standard I/O and redirection. I/O redirection using dup() and dup2()

Lec5.1
SPVL24
SPVL25
 
18

Use of pipes on the shell and in a C program using the pipe() system call. How to use a pipe for IPC among related processes for unidirectional communication. The concept of using two pipes for bidirectional communication. A C program that simulate the shell command man ls | grep ls | wc
Use of named pipes or FIFOs in a C program using the mkfifo() and mknod() system call. How to use a named pipe for IPC among un-related processes for unidirectional communication. The concept of using two named pipes for bidirectional communication. A simple client server application using FIFO.

Lec5.2
SPVL-26
SPVL-27
 
19

Introduction to message queues. Difference between System-V and POSIX message queues. Data structures used by System-V message queues. Use of msgget(), msgsnd(),msgrcv(),msgctl() functions of System-V message queues

Lec5.3
SPVL-28
 
20

Introduction to shared memory: Use of shmget(), shmat(), shmdt(), shmctl() functions of System-V shared memory. Use of ipcs, and ipcrm commands
Introduction to memory mapped files. The location of memory mappings in process address space. Comparison between private vs shared file mapping. Use of mmap(), msync(), and munmap(). Sample programs for proof of concept.

Lec5.4
Lec5.5
SPVL-29
SPVL-30
 
21Module-6 (Synchronization):

Overview of synchronization. The concept of race condition and critical section problems. The concept of direct and indirect data sharing among threads, threads safety and reentrant functions. Introduction to mutex, locking, unlocking and destroying a mutex object. Mutex attributes and mutex types. Introduction to condition variable. The concept of thread cancellation is introduced. Solution to Producer consumer problem and barber shop problem.

Lec6.1
SPVL-31
RPaper-10
RPaper-11
 
22

POSIX semaphores vs System-V semaphores. A comparison between mutex, condition variable and semaphore. The implementation of named and unnamed semaphores and their related API. Sample programs giving a CSP solution using semaphores among threads and processes. The solution of serialization among threads and processes using semaphores. Use of counting semaphores. Working with named semaphores: sem_open(), sem_post(), sem_wait(), sem_close(), sem_unlink(), sem_getvalue(). Working with unnamed semaphores: sem_init(), sem_destroy()

Lec6.2
SPVL-32
 
23Module-7 (Concurrent, Parallel and Distributed Systems):

Introduction to TCP/IP Programming Paradigm. Configuring some basic servers like telnet, dhcp, ssh, and ftp. Introduction to Socket, TLI, MacTCP and winsock Application Programming Interfaces. Introduction to TCP and UDP Client-Server Communication using BSD Socket API
Introduction to Client Server paradigm. Introduction to Socket, TLI, MacTCP and winsock Application Programming Interfaces. A detailed comparison of TCP and UDP sockets with examples. Three way connection establishment and four way connection termination showing practically using Wireshark tool. The system call graph of Internet domain TCP Client and server. The related system calls for creating and managing the sockets. Proof of these concepts with example codes of echo and daytime clients and servers. Some important lookup functions like gethostbyname() and getservbyname() with examples.

Design and code of a web server
Design and code of licensing server

Lec7.1
Lec7.2
SPVL-33
SPVL-34
RPaper-12
RPaper-13
RPaper-14
 
24

Introduction to Datagram Sockets by giving an overview of how datagram sockets work. The system call graph of Internet domain UDP Client and server. The related system calls for creating and managing the sockets. Proof of these concepts with example codes of echo, daytime and time clients and servers.
Introduction to UNIX domain sockets. A comparison between UNIX domain TCP sockets and UNIX domain UDP sockets is made. Proof of these concepts is given by example codes of UNIX domain TCP echo client server programs as well as UNIX domain UDP sender and receiver programs.

Lec7.3
Lec7.4
SPVL-35
SPVL-36
 
25

Difference between iterative and concurrent servers. What happens when multiple clients access an iterative echo server. Design and code of concurrent echo server using

  • Using fork()
  • Using pthread_create()
  • Using select()
Need of concurrent clients

Design and code of a nc(1) command

 

Lec7.5
SPVL-37
 
26

Overview of Remote Procedure Calls, Concept of marshaling and data copying, Kernel copies and copying overhead, making RPC cheap, RPC on Symmetric Multiprocessing Architecture (SMP)

Lec7.6 
27

Overview of Shared memory multiprocessor architecture, Refresher on page fault service, Parallel OS and page fault service, Clustered objects, their implementation and advantages.
Overview of Distributed Systems, Happened before relationship, Identifying events and ordering of events.
Introduction to Lamport logical clocks, Lamport total order, Distributed mutual exclusion lock algorithm, IPC time and clock drift

Lec7.7 
28

Distributed Shared memory
Distributed File Systems
Giant Scale Services
Map Reduce

Lec7.8 
29Module-8 (Cyber Security):

Overview of Cyber Security. Overview of buffer overflow. A classic stack based buffer overflow. How a stack based buffer overflow be exploited? Exploit mitigation techniques. The architecture of x86_64, its assembly and function calling convention. Installing and using PEDA. Changing the control of flow of execution in PEDA.

Lec8.1
SPVL-38
RPaper-15
RPaper-16
 
30

Overview of shell code. Writing your own shell code and using it in a stand alone C-program. Getting shell code from Internet archives and using them in a stand alone C-program. Writing your shell codes using pwn tools and using them in stand alone C-program. Writing your shell codes using msfvenom and using them in stand alone C-program

Lec8.2
SPVL-39
 
31

Finding vulnerabilities in executables. Crafting an input string to vulnerable programs to shift the control of flow of execution to some other part in the code section. Injecting shell code via input string and transferring control of flow to it. Injecting shell code via environment variables and transferring control of flow to it. Issues of exploiting vulnerable programs inside and outside gdb. Exploiting a vulnerable echo server executing on a remote machine and creating a tcp bind shell and a reverse tcp shell. A discussion on bypassing exploit mitigation techniques used by most modern operating systems and compilers.

Lec8.3
SPVL-40
 
32

Hacking a MS Windows machine from Kali Linux using metasploit frame work Implementation of mitigation techniques for buffer over flow vulnerabbility

  
 
Sr.Topic DescriptionLecture ResourceSessional Instruments
1Introduction to course, pre-requisite, policies, tools, and grading system. What is an Operating System (OS) and why it is needed to manage h/w? Operating System services, interrupts, traps and signals. Dual mode operations and protection mechanism. Types of operating systems and computing environmentsLec01.pdf
Linux-Commands-Cheat-Sheet.pdf
VLec00.mp4
 
2Introduction to virtualization and hypervisors. Installing Linux (Ubuntu, Kali, CentOS) on Virtualbox. Introduction to Linux command line interface, Linux File Hierarchy Standard and basic shell commands. Linux system call interface. Compiling a C program on Linux command line interfaceLec02.pdf
VLec01.mp4
VLec02.mp4
VLec03.mp4
 
3Editors used in Linux (vim, peco, nano). Shell commands (wc, sort, uniq, grep, cut, paste, comm, comp, diff, whereis, which, locate, find, tar, gzip, gunzip). Program on disk and its components. Viewing contents of a program file. Process in memory and its components (stack, heap and PCB). Command line arguments and environment variables. Viewing contents of a running program using readelf and objdumpLec03.pdf
VLec04.mp4
VLec05.mp4
 
4CPU and I/O bound processes. Process state models. Five, six and seven state process models. Process scheduling queues. Long term, medium term and short term schedulers. Concept of process/context switch. Process creation and termination. Shell commands related to process management. Fork, wait and exit system calls. Interrupt, trap and system calls. Process resource limits.Lec04-05.pdf 
5Discussion on working of a Linux shell and the concept of how an internal and external command executes. Running programs in the background and foreground. Switching programs to different states. Basic commands related to process states like ps, fg, bg and top.Lec6-7.pdf
VLec06.mp4
VLec07.mp4
 
6The open, read, write and close paradigm in Linux. The concept of PPFDT. The connection of an opened file from process PPFDT to System Wide File Table, to I-node table and finally to disk blocks. Cooperating Processes. Taxonomy of Inter-process Communication  
7I/O Redirection, UNIX IPC tools. Using pipes, FIFOs and signals in LinuxVLec08.mp4
VLec09.mp4
VLec10.mp4
 
8Concurrent and parallel programming. Introduction to threads. Multi-threading. Merits and demerits of threads. User level vs Kernel level threads. Threading models. Programming using Pthread library.Lec8.pdf 
9Process scheduler and Dispatcher. Preemptive vs non-preemptive scheduling. CPU and IO bursts. CPU scheduling and scheduling criteria, FCFS, SJF, SRTF, and Priority scheduling.Lec09-11.pdf 
10Round Robin, Virtual Round Robin, Multi level Queue Scheduling and Multi level Feed- back queue scheduling, Rotating Stair-case Dead line scheduler,  
11Rotating Stair-case dead line scheduler, UNIX SVR3 scheduling algorithm. Changing process priorities using nice, renice commands. Changing nice value of running processes and executing a program with a nice value other than the default. The concept of hard and soft CPU affinity in Linux. Displaying and changing the scheduling parameters of Linux processes using schedtoolVLec11.mp4 
12Introduction to synchronization, Concurrency Control, Race Condition, Critical Section Problem. Concept of atomic operation. General format of a CS problem solution. Characteristics of a good CSP solution.Lec12.pdf
VLec12.mp4
 
13Software Based Solutions to CSP: Dekker solution, Peterson solution, and Leslie Lamport’s Bakery algorithm. Concept of busy waiting.Lec13-14.pdf
VLec13.mp4
 
14H/W based solutions to CSP: Disabling of interrupts, TSL and swap instructions. Thread synchronization using pthread_mutex_t variable and pthread_mutex_lock() and pthread_mutex_unlock() library callsVLec14.mp4 
15Introduction to semaphores. Binary and counting semaphores. Achieving mutual exclusion using semaphores. Achieving serialization using semaphores. Solution to Standard Synchronization problems using semaphores, Producer Consumer, Dining Philosopher, Reader writer, Sleeping Barber, Smokers problemLec15.pdf
VLec15.mp4
 
16Limitations of semaphores, Introduction to Monitors, Condition variables, Hoare and Mesa monitors. Solution to standard synchronization problems using monitors.Lec16.pdf

 
17Introduction to Dead locks, Four necessary and sufficient conditions for Dead Locks, Resource allocation graph, Dead lock handling methods, Dead lock preventionLec17-18.pdf

 
18Dead Lock Avoidance. Bankers and Safety Algorithm.
Dead Lock Detection and Recovery Algorithms
  
19Memory management, address binding and linking, Logical vs Physical addresses, Dynamic loading, Dynamic linking and shared libraries. Overlays, swapping. Introduction to contagious memory allocationLec19-20.pdf

 
20MFT and MVT, Placement algorithms, Internal and External fragmentation, Buddy partitioning scheme.  
21Introduction to paging, Page Table, Address translation in paging, Paging parameters for Intel and PDP11Lec21-23.pdf

 
22Implementing page table in cache, memory and CPU registers, Structure of Page Tables (Hierarchical, Inverted and Hashed Page tables). Introduction to Segmentation, address translation in segmentation.  
23Introduction to paged segmentation, address translation in a paged segmentation. Address translation in Intel 80386 (Real and protected mode)  
24Virtual Memory, Background, Demand Paging, Performance of Demand Paging, Page Replacement algorithms (FIFO, Optimal, LRU, LFU, MFU, Buffering)Lec24-25.pdf

 
25Copy on Write protocol and vfork() system call, Allocation Of Frames, Thrashing, Resident Set Management, Working Set Model, Page fault frequency, memory mapped files.  
26Hard Disk Geometry: Spinning and Solid state disk. Working of spinning disk and its interfaces (IDE, ATA, SATA, SCSI, SAS). The concept of Logical Block Addressing and its mapping on CHS address.
Hard Disk Partitions: Partitioning a hard disk. Different types of partition tables, MBR and GPT. Linux tools used for partitioning a hard disk like fdisk, gdisk, parted, gparted, cfdisk, sfdisk
Lec26.pdf
VLec16.mp4
VLec17.mp4
 
27Disk Formatting: Concept of a file system and the basic functionalities that every file system should offer. Comparison of different file systems like ext2/3/4, reiserfs, hpfs, minix, ntfs, vfat, xfs and zfs. Use of Linux tools like mkfs, mke2fs, mkntfs, mkfs.fat, mkfs.minix to put a file system on a partition
File System Mounting: Introduction to the concept of file system mounting. Linux configuration files related to file system mounting. Linux commands like mount, umount, lsblk, blkid. Maintaining integrity of file system using Linux commands like fsck, e2fsck, fsck.fat, fsck.nfs
Lec27.pdf
VLec18.mp4
VLec19.mp4
 
28File System Architecture: Schematic view of a standard UNIX file system. Describe the contents of boot block, super block, inode block, and data blocks. Discuss In-memory and on-disk structures used by a file system. Describes what actually happens behind the curtain when a user creates, accesses and deletes a file and how Linux keep track of opened files by a process. Use of Linux commands like df, du, lsof, fuser, and tune2fs to perform these tasks
Hard and Soft Links: Discuss the use of hard and soft links on all UNIX based systems. Differences between hard and soft links. Use of Linux command ln to create hard and soft links
Lec28.pdf
VLec20.mp4
VLec21.mp4
 
29File Permissions: Discuss the use of standard file permissions. How to change the existing file permissions on a file using symbolic and octal way. Use of chmod and chown commands. Setting the default file permissions on a newly created file using the umask command
Special File Permissions: Concept and use of Saved SUID bit on files. Concept and use of Saved SGID bit on files and directories. Concept and use of Sticky bit on files and directories.
Lec29.pdf
VLec22.mp4
VLec23.mp4
 
30Access Control Lists: Discuss the security on files using Access Control List. Concept of Discretionary Access control and Mendatory Access control. How to set ACLs on files. A discussion on default ACLs or ACLs on directories
Device Files: Seven File Types in Linux and the concept of device files. Describes the contents of /dev/ directory. Describes Major and minor numbers and shows how you can create your own device files. Important /dev/ directory files like zero, null, full, random, urandom and ttys
Lec30.pdf
VLec24.mp4
VLec25.mp4
 
31Terminal Attributes: Overview of Terminal Devices and a comparison between disk and terminal files. Examine current attributes of terminal driver on a Linux machine and changing them using stty command. Overview of Canonical and Non-canonical mode of terminal drivers.
Time Management in Linux operating system
Managing services using systemd: Introduction to Linux system daemon. Overview of to systemd unit files, specially Target Unit Files and Service Unit Files. Shell commands to manage services using systemctl. Writing/running a basic service of your own
VLec26.mp4
VLec27.mp4
VLec28.mp4
 
32Log Files: Logging mechanism in Linux
Booting process of a Linux system: A discussion on five phases of Linux Operating system: BIOS / UEFI Initialization, Master Boot Record, Boot Loader, Kernel Initialization, init or systemd Process

VLec29.mp4
VLec30.mp4
 
 
Sr.Topic DescriptionLecture ResourceSessional Instruments
1

Module-1 (Preparing your toolbox):
Discussion on Course Matrix. Application vs system programming perspective. How system calls work (Behind the curtain).

Cheatsheet
Lec01.pdf
SPVL-01
 
2

C-Compilation process from system programmer perspective. Making a system call with and without wrapper. Making a system call from within Assembly program. (Tools used are are gcc, gdb, readelf, objdump, nasm)

Lec02.pdf
SPVL-02
 
3

Linking and loading a process (Behind the curtain). Load time and run time dynamic linking. Structure of ELF file format. Merging re-locatable object files into executable. What is relocation. Symbol (global, external, local) resolution. Strong and weak symbols. Linker symbol rules for multiple strong and weak symbols. Creating and using your own static libraries. Creating and using your own dynamic libraries or shared objects. (Tools used are nm, ar, ranlib)

Lec03.pdf
SPVL-03
 
4

Introduction to make utility, how make works, Makefile rules and targets. Using your own and built-in macros in a Makefile. Writing and calling multiple make files for a program. Comparison between binary and open source software packages. Downloading and installing open source software. Packaging your own software using GNU autotools (autoconf and automake). Packaging your own software using cmake utility. Writing your own man pages. (Tools used are make, autoconf, automake, cmake, cpack)

Lec04.pdf
Lec05.pdf
SPVL-04
SPVL-05
 
5

Local version control systems (SCCS, RCS). Centralized version control systems (CVS, SVN). Distributed version control systems (git, bitkeeper, darcs, mercurial) Overview of git. Downloading and installing git. Basic git workflow and its configuration. Initializing git repository, Adding, editing, deleting and renaming files, and viewing commit log. Ignoring files in git and undoing changes.
Overview of git branches. Creating, renaming, deleting and comparing branches. Overview of merging branches. The concept of fast forward and real merge. Handling merge conflicts. Concept of remote repositories and creating one on bitbucket. Pushing a local repo to bitbucket. Cloning an existing repo from bitbucket

Lec06.pdf
Lec07.pdf
SPVL-06
SPVL-07
 
6

How a C program starts and terminates. Normal vs abnormal termination. Registering exit handlers using atexit() and on_exit() functions. Querying process resource limits and changing them on the shell. Getting and setting process resource limits from within a C program using getrlimit() and setrlimit() functions.

Lec08.pdf
SPVL-08
 
7

Memory layout of a C program. Use of command line arguments and environment variables in C programs. A discussion on layout of process stack and how it grows and shrinks. Stack buffer overflow problem. Doing a non-local goto using longjmp() and setjmp()

Lec09.pdf
SPVL-09
 
8

Allocating using and freeing memory on heap. Layout of heap and heap allocators. Role of system calls brk() and sbrk() in management of heap. Common programming errors related to heap management. Tools and libraries for malloc debugging like splint, electric-fence, and valgrind

Lec10.pdf
SPVL-10
 
9

Module-2 (File, Information and Time Management):

Hard disk partitioning, formatting and mounting. Internal structure of UNIX file system (Disk, partitions, boot block, super block, inode block). Connection to an open file via PPFDT. Universal I/O model (open-read-write-close paradigm). Kernel Buffering of file I/O and Buffering in the stdio library. Misc important file and directory related system calls

Lec11.pdf
Lec12.pdf
SPVL-12
 
10

Different techniques of I/O redirection. Use of fcntl() system call to duplicate a file descriptor, get/set file descriptor flags in PPFDT, get/set file status flags in SWFT, and achieving locks on files.

Lec13.pdf
SPVL-13
 
11

Directory management in C Programs. Working of UNIX ls utility. How it works? Can we design ls utility of our own?

(Design and code of UNIX ls utility)

 

Lec14.pdf
SPVL-14
 
12

(Design and Code of UNIX who utility)

Lec15.pdf
SPVL-15
 
13Module-3 (Process Management and Scheduling):

Process Identifiers: getpid(), getppid(), getuid(), setuid(), getgid(), setgid(), geteuid(), seteuid(), getegid(), setegid(). Process creation using fork(), vfork(), clone(). Copy on Write Semantics. Orphan and Zombie process

Lec16.pdf
Lec17.pdf
SPVL-17
Linux-Kernel-Compilation-1
14

Process trees, chains and fans. Monitoring Child Processes by accessing termination status of the child process using wait(), waitpid(), waitid(), wait3(), wait4() and WIFEXITED(), WEXITSTATUS(), WIFSIGNALED() and WTERMSIG() macros

Lec18.pdf
SPVL-18
 
15

Six exec family functions execl(), execle(), execlp(), execv(),execve(), execvp(). Process Groups, Process Sessions, and concept of Controlling Terminals. Impact of fork() and exec() on different process attributes. writing your our own system() function and use it to create a shell utility

Lec19.pdf
SPVL-19
 
16

Overview of daemon processes in Linux. Writing your own daemon process programatically. Introduction to systemd (replacement of SysV init daemon). Controlling daemons using systemctl utility. Writing long lived process and managing it using systemctl.

Design and Code your own daemon using systemd.

Design and code of Shell utility (SPVL-22)

Lec20.pdf
Lec21.pdf
Lec22.pdf
SPVL-20
SPVL-22
 
17

Different ways to achieve concurrency. Conventional Concurrent Servers and their Problems. Thread implementation models. Linux implementation of POSIX threads (LinuxThreads and NPTL). Thread creation and termination, joining threads, passing implicit data types to thread function, and passing structures to thread function. Returning and receiving data from a thread function. Creating Thread arrays on stack, and creating thread arrays on heap. Thread attributes (detachstate, stackaddr, stacksize, priority, inheritsched). Changing the default attributes of a thread. Threads and signals. Threads and fork(). Thread cancellation.

Lec23.pdf
SPVL-23
 
18Module-4 (Inter-Process Communication):):

Introduction to Linux Inter-Process communication tools. Overview of Standard and Real time Signals. Signal handling on Bash shell using kill and top utilities. Sending signals using (kill, raise, abort, pause, alarm, sleep, usleep, nanosleep) functions

Lec24.pdf
SPVL-24
 
19

Ignoring and handling signals using signal() system call. Avoiding race conditions using signal mask and sigprocmask() system call. Limitations of signal() system call. Ignoring and handling signals using sigaction() system call. Scheduling future actions using interval timers getitimer(), setitimer()
Facts about standard I/O and redirection. I/O redirection using dup() and dup2()

Lec25.pdf
SPVL-25
 
20

Use of pipes on the shell and in a C program using the pipe() system call. How to use a pipe for IPC among related processes for unidirectional communication. The concept of using two pipes for bidirectional communication. A C program that simulate the shell command man ls | grep ls | wc

Lec26.pdf
SPVL-26
 
21

Use of named pipes or FIFOs in a C program using the mkfifo() and mknod() system call. How to use a named pipe for IPC among un-related processes for unidirectional communication. The concept of using two named pipes for bidirectional communication. A simple client server application using FIFO.

Lec27.pdf
SPVL-27
 
22Module-5 (Thread Management and Synchronization):

Overview of synchronization. The concept of race condition and critical section problems. The concept of direct and indirect data sharing among threads, threads safety and reentrant functions. Introduction to mutex, locking, unlocking and destroying a mutex object. Mutex attributes and mutex types. Introduction to condition variable. The concept of thread cancellation is introduced.

Lec31.pdf
SPVL-31
 
23Module-6 (Network Programming):

Introduction to TCP/IP Programming Paradigm. Configuring some basic servers like telnet, dhcp, ssh, and ftp. Introduction to Socket, TLI, MacTCP and winsock Application Programming Interfaces. Introduction to TCP and UDP Client-Server Communication using BSD Socket API

Lec33.pdf
SPVL-33
 
24

Flow chart showing TCP and UDP based client server application using socket(), bind(), listen(), connect(), accept(), write(), send(), sendto(), read(), recv(), recvfrom(), and close().Three way connection establishment and four way connection termination.The related system calls for creating and managing the sockets.
Writing a helloworld tcp client server application

Lec34.pdf
SPVL-34
 
25

Proof of these concepts with example codes of echo and daytime clients and servers. Some important lookup functions like gethostbyname() and getservbyname() with examples.Host vs NW Byte order (htons(), ntohs(), htonl(), ntohl()).

Lec34.pdf
SPVL-34
 
26

Introduction to Datagram Sockets by giving an overview of how datagram sockets work. The system call graph of Internet domain UDP Client and server. The related system calls for creating and managing the sockets. Proof of these concepts with example codes of echo, daytime and time clients and servers. Look up functions (gethostbyname(), getservbyname(), getprotobyname())

Design and code of a web server

Lec35.pdf
Lec36.pdf
Lec37.pdf
 
27Module-7 (Network Security):

Overview of Cyber Security. Overview of buffer overflow. A classic stack based buffer overflow. How a stack based buffer overflow be exploited? Exploit mitigation techniques. The architecture of x86_64, its assembly and function calling convention. Installing and using PEDA. Changing the control of flow of execution in PEDA.

Lec38.pdf
SPVL-38
 
28

Overview of shell code. Writing your own shell code and using it in a stand alone C-program. Getting shell code from Internet archives and using them in a stand alone C-program.

Lec39.pdf
SPVL-39
 
29

Writing your shell codes using pwn tools and using them in stand alone C-program. Writing your shell codes using msfvenom and using them in stand alone C-program

Lec39.pdf
SPVL-39
 
30

Finding vulnerabilities in executables. Crafting an input string to vulnerable programs to shift the control of flow of execution to some other part in the code section. Injecting shell code via input string and transferring control of flow to it. Injecting shell code via environment variables and transferring control of flow to it. Issues of exploiting vulnerable programs inside and outside gdb. Exploiting a vulnerable echo server executing on a remote machine and creating a tcp bind shell and a reverse tcp shell.

Lec40.pdf
SPVL-40
 
 
Sr.Topic DescriptionLecture ResourceSessional Instruments
1Introduction to Computer Organization and Assembly Language Programming and a discussion on the course matrix. The course will be having two parts in the first half of the course we will be designing and writing the HDL of a full-blown computer, will design its machine and assembly language, will write programs and execute those programs on the designed h/w architecture. At the end of the first half of the course, we will also design and write an Assembler (in C) for the designed computer. The second half of the course will deal with the assembly of the all-time famous x86-64 architecture. The links from where the e-books, tools, code snippets, and lecture slides, and other misc resources can be downloaded are mentioned.Book-01
Book-02
Book-03
Book-04
Lec-01.pdf
VL-01
Tools and Resources (Bitbucket repo)
2HDL for Combinational Circuits – I:
Review of Boolean logic and gates. Introduction to Hardware Description Languages. Design and code of And, Or, Not gates using the universal NAND gate. Downloading and installing Hardware Simulator and interactive chip testing on this simulator. Designing Xor chip and performing script based testing of Xor chip on h/w simulator.
HDL for Combinational Circuits – II:
Design and HDL code for XOR chip, using And, OR, Not gate chips. A demo of Verification of XOR chip using interactive chip testing in the h/w simulator. A brief overview of script-based chip testing. Writing script for testing of the designed XOR chip. A discussion on key players involved in a hardware construction project.
Lec-02.pdf
Lec-03.pdf
VL-02
VL-03
 
3HDL for Combinational Circuits – III:
Design and HDL code for some standard combinational circuits like Encoder, Decoders, Multiplexers, and De-Multiplexer chips. A demo of Verification / Testing of these standard combinational chips using interactive chip testing in the h/w simulator.
HDL for Combinational Circuits – IV:
Design and HDL code for multi-bit gates. The concept of buses and the design of chips having buses as input. Design and code of And16, Or16, Not16, and Mux16 chips having 16-bits inputs. Design and code of And4way16 and similar chips having four inputs with each input of 16 bits.
Lec-04.pdf
Lec-05.pdf
VL-04
VL-05
 
4Data Storage – I:
Data Representation in Computers, Unsigned, and Signed Numbers, Sign magnitude representation and its limitations, 1s Complement representation and its limitations, 2s Complement representation, Comparisons and pros and cons of each, Ranges and different Storage Sizes, Overflow in Unsigned and Signed Numbers, How the Hardware Detect an Overflow, Concept of Sign Extension, Encoding Characters and Strings (ASCII and Unicode)
Lec-06.pdf
VL-06
 
5Data Storage – II:
Encoding Real Numbers, Fixed Point Representation, Floating Point Representations (IEEE-754), Storage layout, Conversion Examples, Range and Precision, Arithmetic Operations, Overflow and Underflow, IEEE-754 Special Values
Lec-07.pdf
VL-07
 
6Design of ALU – I:
Review of HDL for Combinational Circuits, Designing a single bit Logic Unit Writing HDL for Combinational Arithmetic Circuits like Half Adder, Full Adder, Full Subtractor, 16-bit Binary Adder (Add16 chip), 16-bit Incrementer (Inc16 chip), Demo of above chips on H/W Simulator
Design of ALU – II:
Components of a Computer System, Design of ALU, The Hack ALU, The Hack ALU Operations, Design of Hack ALU, HDL of Hack ALU, Verifying the ALU chip on H/W Simulator
Lec-08.pdf
Lec-09.pdf
VL-08
VL-09
 
7Design of Sequential Circuits:
Why Sequential Circuits? Understanding Time in Circuits, Combinational vs Sequential Circuits, Flip Flops, D flip Flop, SR Flip Flop, JK Flip Flop, T Flip Flop
Design of Registers:
What are Registers, Design of 1-bit Register, HDL for 1-bit Register, Design of 16-bit Register, HDL for 16-bit Register
Lec-10.pdf
Lec-11.pdf
VL-10
VL-11
 
8Design of Memory:
Concept of Memory Hierarchy, Multi-Byte Read/Write, Design of Random Access Memory, Read/Write Logic of RAM, API of a RAM Chip, HDL of 8 Words RAM, HDL of 64 Words RAM, HDL of 512 Words RAM, HDL of 4K Words RAM, HDL of 16K Words RAM
Design of Counters:
Overview of Hack Computer Components, Overview of Counters, Why do we need Counter for our Hack Computer, Concept of Program Counter, Counter Simulation, Design and Implementation of PC for Hack Computer, Demo on H/W Simulator
Lec-12.pdf
Lec-13.pdf
VL-12
VL-13
 
9ISA-I:
Overview of Computer System, Universality of Computer System, Turing Machine, Von Neumann Architecture, Instruction Set Architecture (ISA)
Lec-14.pdf
VL-14
 
10ISA-II:
Five Dimensions of ISA, Class of ISA, Types and Sizes of Operands, Operations (including control flow instructions), Memory Addressing Models and Addressing Modes, Encoding an ISA
Lec-15.pdf
VL-15
 
11Hack Machine Language – I:
Hack Computer Machine Language, Review of h/w of Hack Computer, Software of Hack Computer, A Instruction, C Instruction, Examples
Lec-16.pdf
VL-16
 
12Hack Machine Language – II:
Review of Hack Symbolic Machine Instructions, A Instruction, C Instruction, Binary Code Format of Hack Computer Instruction, Encoding of 16 bit A-Instruction, Encoding of 16 bit C-Instruction, Examples, A Complete Hack Program: Assembly Language
Lec-17.pdf
VL-17
 
13Interfacing I/O Devices:
How to interface I/O devices with computer, Interfacing Screen with Hack computer, Demo of built-in Screen chip on h/w Simulator, Interfacing Keyboard with Hack computer, Demo of built-in Keyboard chip on h/w Simulator
Lec-18.pdf
VL-18
 
14Hack Assembly Programming – I:
Review of Hack Computer Assembly Instructions, Hack Assembly Programs, A Hello World in Hack assembly, CPU Emulator, Demo, Program Termination
Hack Assembly Programming – II:
Recap previous lecture, Symbols in Hack Assembly Language, Built-in Symbols, Label Symbols, Variable Symbols, Branching, Iteration
Lec-19.pdf
Lec-20.pdf
VL-19
VL-20
 
15Hack Assembly Programming – III:
Review of Hack Assembly Programs, Pointers and Arrays, Input / Output Instructions, Debugging, Review of Hack Assembly Programs, Pointers and Arrays, Input / Output Instructions, Debugging
Lec-21.pdf
VL-21
 
16Data path of Hack CPU-I:
Von Neumann Architecture, Flow of Information inside Computers, Buses, Data Bus, Address Bus, Control Bus, Fetch Execute Cycle, Fetch Execute Clash, Harvard Architecture
Data path of Hack CPU-II:
Review of Hack Computer Architecture, Hack CPU Interface, Hack CPU Implementation, Input/output and Operations of Hack ALU, Control Logic of Hack CPU
Lec-22.pdf
Lec-23.pdf
VL-22
VL-23
 
17Design of Hack Computer:
Recap of Hack Computer Architecture, Implementation of Hack CPU Chip (CPU.hdl), Implementation of Hack Memory Chip (Memory.hdl), RAM16 chip (RAM16K.hdl), Screen chip (Screen.hdl), Keyboard chip (Keyboard.hdl), Implementation of Hack ROM Chip (ROM32K.hdl), Implementation of Hack Computer Chip (Computer.hdl)
Lec-24.pdf
VL-24
 
18Design of Hack Assembler:
What is an Assembler? How an Assembler works? Hack Machine Language Specification, Demo of Built-in Hack Assembler, Design of Hack Assembler (w/o Symbols), Design of Hack Assembler (with Symbols), Hack Assembler Implementation in C/C , Executing Hack Machine Code, Hack Computer Chip in h/w Simulator, CPU Emulator
Lec-25.pdf
VL-25
 
19History and Evolution of Intel Microprocessors:
Intel 4004 (1971), Intel 8008, Intel 8080, Intel 8086 (x86), Intel 80286, Intel 80386, Intel 80486, Intel 80586 (Pentium P5), Intel 80686 (Pentium P6), Intel Core (2006)
Intel Nehalam (2008), Intel Sandy Bridge, Intel Ivy Bridge, Intel Haswell, Intel Broadwell, Intel Sky Lake, Intel Kaby Lake, Intel Coffee Lake, Intel Coffee Lake Refresh, Intel Comet Lake (2019)
On Improving Processors Performance:
CPU Performance Equation, Single Cycle vs Multi Cycle CPU Architecture, Pipelined CPU Architecture, Pipeline Stages, Even vs Uneven pipelined stages, Pipelined Hazards, Solutions of Pipeline Hazards, CISC vs RISC Architecture
Lec-26.pdf
Lec-27.pdf
VL-26
VL-27
 
20Programming Model of x86 Architecture:
Layout of memory models (flat, segmented) and register set file of Intel 8080, 80386, x86-64. Logical to physical address translation for segmented memory model.
Lec-28.pdf
VL-28
 
21Hello World in x86-64 Assembly:
Overview of microprocessor families and their corresponding assembly languages. Tool chain and programming environment for x86-64 assembly programming. Running the first hello world assembly program.
Structure of x86-64 Assembly Program:
A discussion on the x86-64 assembly language instruction format and the overall structure of assembly program including the contents of .data, .bss and .text section. A scuba diving in the sea of how a system call is made.
Lec-29.pdf
Lec-30.pdf
VL-29
VL-30
 
22Debugging C-Program with gdb:
A review of C-compilation process. What is a debugger? Why use gdb? How to compile, load and run a program inside gdb and get information about the running process. Getting help inside gdb, setting break points, watch points, stepping through the code, examining and modifying variables, convenience variables and setting conditional break points.
Data Types and Endianness in x86-64:
Usage of different data types and special tokens in NASM. Practically understand about the endianness of a machine
Lec-31.pdf
Lec-32.pdf
VL-31
VL-32
 
23Data Transfer Instructions and Process Stack:
Usage of different move instructions like mov, movzx, movsx, lea and xchg. A discussion on the working of process stack and the push and pop instructions.
Memory Addressing Modes:
Theoretical concepts and pros and cons of addressing modes used by different processors. Addressing modes used by x86-64 like Base-Index-Sale-Displacement
Lec-33.pdf
Lec-34.pdf
VL-33
VL-34
 
24Arithmetic Instructions Part-I:
A recap of x86-64 register set and the programming tool chain. Summary of major categories of x86-64 instructions. A practical demo on the use of add, adc, sub, sbb, inc, dec, neg, cmp, clc, stc, and cmc. A discussion on how the flags are effected after these arithmetic instructions
Arithmetic Instructions Part-II:
A recap of x86-64 register set and the programming tool chain. Summary of major categories off x86-64 instructions. A practical demo on the use of mul, div, imul, idiv instructions. A discussion on how the flags are effected after these arithmetic instructions
Lec-35.pdf
Lec-36.pdf
VL-35
VL-36
 
25Logical Operations:
A recap of x86-64 register set and the programming tool chain. Summary of major categories of x86-64 instructions. A practical demo on the use of and, or, not, xor, and test instructions. A discussion on how the flags are effected after these logical instructions.
Bit-Shifting Operations:
A recap of x86-64 register set and the programming tool chain. Summary of major categories of x86-64 instructions. A practical demo on the use of shl, sal, shr, sar, rol, ror, rcl, and rcr instructions. A discussion on how the flags are effected after these logical instructions.
Lec-37.pdf
Lec-38.pdf
VL-37
VL-38
 
26Control Instructions – I:
A discussion on control of flow of execution of a program and how to change it. Description of unconditional jump instruction with a demonstration of example programs. Discussion on signed and unsigned conditional jump instructions with demonstration of example programs. Translating if…else code too assembly language.
Control Instructions – II:
A Recap of previous session. Translating high level repetition structure (for, while, ….) to its corresponding x86 assembly code using conditional jump instructions as well as using x86 loop instructions.
Lec-39.pdf
Lec-40.pdf
VL-39
VL-40
 
27GDB with PEDA Plugin:
A recap of gdb command line and text user interface mode. Downloading, installing, and configuring Python Exploit Development Assistance (PEDA) plugin to enhance the firepower of gdb. Debugging the x86-64 assembly programs using gdb with PEDA and a brief intro of using this plugin for reverse engineering and exploit development
Functions in Assembly Language – I:
What are functions? Why they are used in programming languages? Syntax of defining an assembly function in NASM and MASM. Understanding the working of x86-64 call and ret instruction. The usage of process run-time stack in function call. The concept and requirement of caller-saved and callee-saved registers
Lec-41.pdf
Lec-42.pdf
VL-41
VL-42
 
28Functions in Assembly Language – II:
Recap of functions in assembly language and use of call and ret instructions. How different operating systems allow x86-64 assembly programmers to pass and return values to and from functions. Designing an assembly function to display a single digit and a multi-digit decimal number on screen. Writing, assembling, linking and executing multi-file assembly programs on x86-64.
Function Calling Convention and FSF:
Recap of assembly language functions. Understanding function calling in high-level languages like C and C . The concept of Function Stack Frame (FSF) or Activation Record used to store data associated with a high-level function on the process run time stack. The x86-64 procedure prolog and procedure epilog for creating and removing FFSF from the stack. A demonstration of stack-based buffer overflow vulnerability and concept of exploiting it.
Lec-43.pdf
Lec-44.pdf
VL-43
VL-44
 
29Mixing C with x86-64 Assembly:
Recap of pushing and popping FSF to and from the process run time stack in high level languages. Calling C-Library functions from within an assembly program. Doing the reverse, i.e., Calling assembly functions from within a C program.
Getting User Input:
User input via system calls, library calls, and command line arguments. What are command line arguments and why we use them in high level languages as well as in assembly programming? Converting the string input received via command line to integer for further processing.
Lec-45.pdf
Lec-46.pdf
VL-45
VL-46
 
30Arrays:
Array address computation (pointer arithmetic) based on its types (bytes, words, double words and quad-words), General pattern for memory references, allocation arrays using malloc, processing arrays, filling with random numbers, printing array elements
String:
Understanding C-strings in assembly, x86 string instructions to store strings in memory, load strings from memory, comparing strings, and scanning strings for substrings
Lec-47.pdf
Lec-48.pdf
VL-47
VL-48
 
31Floating Point Instructions:
8087 floating point instructions that use stack of 80 bit floating point registers (ST0, ST1, …). Intel Core iseries floating point instructions that work with Streaming SIMD Extensions (SSE) 128-bit registers (xmm0, xmm1, …). The concept of Advanced Vector Extensions (AVX) that use 256-bit registers (ymm0, ymm1, …)
Data movement instructions (movss, movsd)
Arithmetic instructions (addss, addsd, subss, subsd, mulss, mulsd, divss, divsd)
Integer / floating point conversion instructions (cvtss2sd, cvtsd2ss, cvtss2si, cvtsi2ss, cvtsd2si, cvtsi2sd)
Floating point control instructions (ucomiss, ucomisd)
Floating point calling convention
Lec-49.pdf
VL-49
 
32Computer Performance and Parallel Processing Hardware:Lec-50.pdf
VL-50
 
 
Sr.Topic DescriptionLecture ResourceSessional Instruments
1Introduction to Computer Organization and Assembly Language Programming and a discussion on the course matrix. The course will be having two parts in the first half of the course we will be designing and writing the HDL of a full-blown computer, will design its machine and assembly language, will write programs and execute those programs on the designed h/w architecture. At the end of the first half of the course, we will also design and write an Assembler (in C) for the designed computer. The second half of the course will deal with the assembly of the all-time famous x86-64 architecture. The links from where the e-books, tools, code snippets, and lecture slides, and other misc resources can be downloaded are mentioned.Book-01
Book-02
Book-03
Book-04
Lec-01.pdf
VL-01
Tools and Resources (Bitbucket repo)
2HDL for Combinational Circuits – I:
Review of Boolean logic and gates. Introduction to Hardware Description Languages. Design and code of And, Or, Not gates using the universal NAND gate. Downloading and installing Hardware Simulator and interactive chip testing on this simulator. Designing Xor chip and performing script based testing of Xor chip on h/w simulator.
HDL for Combinational Circuits – II:
Design and HDL code for XOR chip, using And, OR, Not gate chips. A demo of Verification of XOR chip using interactive chip testing in the h/w simulator. A brief overview of script-based chip testing. Writing script for testing of the designed XOR chip. A discussion on key players involved in a hardware construction project.
Lec-02.pdf
Lec-03.pdf
VL-02
VL-03
 
3HDL for Combinational Circuits – III:
Design and HDL code for some standard combinational circuits like Encoder, Decoders, Multiplexers, and De-Multiplexer chips. A demo of Verification / Testing of these standard combinational chips using interactive chip testing in the h/w simulator.
HDL for Combinational Circuits – IV:
Design and HDL code for multi-bit gates. The concept of buses and the design of chips having buses as input. Design and code of And16, Or16, Not16, and Mux16 chips having 16-bits inputs. Design and code of And4way16 and similar chips having four inputs with each input of 16 bits.
Lec-04.pdf
Lec-05.pdf
VL-04
VL-05
 
4Data Storage – I:
Data Representation in Computers, Unsigned, and Signed Numbers, Sign magnitude representation and its limitations, 1s Complement representation and its limitations, 2s Complement representation, Comparisons and pros and cons of each, Ranges and different Storage Sizes, Overflow in Unsigned and Signed Numbers, How the Hardware Detect an Overflow, Concept of Sign Extension, Encoding Characters and Strings (ASCII and Unicode)
Lec-06.pdf
VL-06
 
5Data Storage – II:
Encoding Real Numbers, Fixed Point Representation, Floating Point Representations (IEEE-754), Storage layout, Conversion Examples, Range and Precision, Arithmetic Operations, Overflow and Underflow, IEEE-754 Special Values
Lec-07.pdf
VL-07
 
6Design of ALU – I:
Review of HDL for Combinational Circuits, Designing a single bit Logic Unit Writing HDL for Combinational Arithmetic Circuits like Half Adder, Full Adder, Full Subtractor, 16-bit Binary Adder (Add16 chip), 16-bit Incrementer (Inc16 chip), Demo of above chips on H/W Simulator
Design of ALU – II:
Components of a Computer System, Design of ALU, The Hack ALU, The Hack ALU Operations, Design of Hack ALU, HDL of Hack ALU, Verifying the ALU chip on H/W Simulator
Lec-08.pdf
Lec-09.pdf
VL-08
VL-09
 
7Design of Sequential Circuits:
Why Sequential Circuits? Understanding Time in Circuits, Combinational vs Sequential Circuits, Flip Flops, D flip Flop, SR Flip Flop, JK Flip Flop, T Flip Flop
Design of Registers:
What are Registers, Design of 1-bit Register, HDL for 1-bit Register, Design of 16-bit Register, HDL for 16-bit Register
Lec-10.pdf
Lec-11.pdf
VL-10
VL-11
 
8Design of Memory:
Concept of Memory Hierarchy, Multi-Byte Read/Write, Design of Random Access Memory, Read/Write Logic of RAM, API of a RAM Chip, HDL of 8 Words RAM, HDL of 64 Words RAM, HDL of 512 Words RAM, HDL of 4K Words RAM, HDL of 16K Words RAM
Design of Counters:
Overview of Hack Computer Components, Overview of Counters, Why do we need Counter for our Hack Computer, Concept of Program Counter, Counter Simulation, Design and Implementation of PC for Hack Computer, Demo on H/W Simulator
Lec-12.pdf
Lec-13.pdf
VL-12
VL-13
 
9ISA-I:
Overview of Computer System, Universality of Computer System, Turing Machine, Von Neumann Architecture, Instruction Set Architecture (ISA)
Lec-14.pdf
VL-14
 
10ISA-II:
Five Dimensions of ISA, Class of ISA, Types and Sizes of Operands, Operations (including control flow instructions), Memory Addressing Models and Addressing Modes, Encoding an ISA
Lec-15.pdf
VL-15
 
11Hack Machine Language – I:
Hack Computer Machine Language, Review of h/w of Hack Computer, Software of Hack Computer, A Instruction, C Instruction, Examples
Lec-16.pdf
VL-16
 
12Hack Machine Language – II:
Review of Hack Symbolic Machine Instructions, A Instruction, C Instruction, Binary Code Format of Hack Computer Instruction, Encoding of 16 bit A-Instruction, Encoding of 16 bit C-Instruction, Examples, A Complete Hack Program: Assembly Language
Lec-17.pdf
VL-17
 
13Interfacing I/O Devices:
How to interface I/O devices with computer, Interfacing Screen with Hack computer, Demo of built-in Screen chip on h/w Simulator, Interfacing Keyboard with Hack computer, Demo of built-in Keyboard chip on h/w Simulator
Lec-18.pdf
VL-18
 
14Hack Assembly Programming – I:
Review of Hack Computer Assembly Instructions, Hack Assembly Programs, A Hello World in Hack assembly, CPU Emulator, Demo, Program Termination
Hack Assembly Programming – II:
Recap previous lecture, Symbols in Hack Assembly Language, Built-in Symbols, Label Symbols, Variable Symbols, Branching, Iteration
Lec-19.pdf
Lec-20.pdf
VL-19
VL-20
 
15Hack Assembly Programming – III:
Review of Hack Assembly Programs, Pointers and Arrays, Input / Output Instructions, Debugging, Review of Hack Assembly Programs, Pointers and Arrays, Input / Output Instructions, Debugging
Lec-21.pdf
VL-21
 
16Data path of Hack CPU-I:
Von Neumann Architecture, Flow of Information inside Computers, Buses, Data Bus, Address Bus, Control Bus, Fetch Execute Cycle, Fetch Execute Clash, Harvard Architecture
Data path of Hack CPU-II:
Review of Hack Computer Architecture, Hack CPU Interface, Hack CPU Implementation, Input/output and Operations of Hack ALU, Control Logic of Hack CPU
Lec-22.pdf
Lec-23.pdf
VL-22
VL-23
 
17Design of Hack Computer:
Recap of Hack Computer Architecture, Implementation of Hack CPU Chip (CPU.hdl), Implementation of Hack Memory Chip (Memory.hdl), RAM16 chip (RAM16K.hdl), Screen chip (Screen.hdl), Keyboard chip (Keyboard.hdl), Implementation of Hack ROM Chip (ROM32K.hdl), Implementation of Hack Computer Chip (Computer.hdl)
Lec-24.pdf
VL-24
 
18Design of Hack Assembler:
What is an Assembler? How an Assembler works? Hack Machine Language Specification, Demo of Built-in Hack Assembler, Design of Hack Assembler (w/o Symbols), Design of Hack Assembler (with Symbols), Hack Assembler Implementation in C/C , Executing Hack Machine Code, Hack Computer Chip in h/w Simulator, CPU Emulator
Lec-25.pdf
VL-25
 
19History and Evolution of Intel Microprocessors:
Intel 4004 (1971), Intel 8008, Intel 8080, Intel 8086 (x86), Intel 80286, Intel 80386, Intel 80486, Intel 80586 (Pentium P5), Intel 80686 (Pentium P6), Intel Core (2006)
Intel Nehalam (2008), Intel Sandy Bridge, Intel Ivy Bridge, Intel Haswell, Intel Broadwell, Intel Sky Lake, Intel Kaby Lake, Intel Coffee Lake, Intel Coffee Lake Refresh, Intel Comet Lake (2019)
On Improving Processors Performance:
CPU Performance Equation, Single Cycle vs Multi Cycle CPU Architecture, Pipelined CPU Architecture, Pipeline Stages, Even vs Uneven pipelined stages, Pipelined Hazards, Solutions of Pipeline Hazards, CISC vs RISC Architecture
Lec-26.pdf
Lec-27.pdf
VL-26
VL-27
 
20Programming Model of x86 Architecture:
Layout of memory models (flat, segmented) and register set file of Intel 8080, 80386, x86-64. Logical to physical address translation for segmented memory model.
Lec-28.pdf
VL-28
 
21Hello World in x86-64 Assembly:
Overview of microprocessor families and their corresponding assembly languages. Tool chain and programming environment for x86-64 assembly programming. Running the first hello world assembly program.
Structure of x86-64 Assembly Program:
A discussion on the x86-64 assembly language instruction format and the overall structure of assembly program including the contents of .data, .bss and .text section. A scuba diving in the sea of how a system call is made.
Lec-29.pdf
Lec-30.pdf
VL-29
VL-30
 
22Debugging C-Program with gdb:
A review of C-compilation process. What is a debugger? Why use gdb? How to compile, load and run a program inside gdb and get information about the running process. Getting help inside gdb, setting break points, watch points, stepping through the code, examining and modifying variables, convenience variables and setting conditional break points.
Data Types and Endianness in x86-64:
Usage of different data types and special tokens in NASM. Practically understand about the endianness of a machine
Lec-31.pdf
Lec-32.pdf
VL-31
VL-32
 
23Data Transfer Instructions and Process Stack:
Usage of different move instructions like mov, movzx, movsx, lea and xchg. A discussion on the working of process stack and the push and pop instructions.
Memory Addressing Modes:
Theoretical concepts and pros and cons of addressing modes used by different processors. Addressing modes used by x86-64 like Base-Index-Sale-Displacement
Lec-33.pdf
Lec-34.pdf
VL-33
VL-34
 
24Arithmetic Instructions Part-I:
A recap of x86-64 register set and the programming tool chain. Summary of major categories of x86-64 instructions. A practical demo on the use of add, adc, sub, sbb, inc, dec, neg, cmp, clc, stc, and cmc. A discussion on how the flags are effected after these arithmetic instructions
Arithmetic Instructions Part-II:
A recap of x86-64 register set and the programming tool chain. Summary of major categories off x86-64 instructions. A practical demo on the use of mul, div, imul, idiv instructions. A discussion on how the flags are effected after these arithmetic instructions
Lec-35.pdf
Lec-36.pdf
VL-35
VL-36
 
25Logical Operations:
A recap of x86-64 register set and the programming tool chain. Summary of major categories of x86-64 instructions. A practical demo on the use of and, or, not, xor, and test instructions. A discussion on how the flags are effected after these logical instructions.
Bit-Shifting Operations:
A recap of x86-64 register set and the programming tool chain. Summary of major categories of x86-64 instructions. A practical demo on the use of shl, sal, shr, sar, rol, ror, rcl, and rcr instructions. A discussion on how the flags are effected after these logical instructions.
Lec-37.pdf
Lec-38.pdf
VL-37
VL-38
 
26Control Instructions – I:
A discussion on control of flow of execution of a program and how to change it. Description of unconditional jump instruction with a demonstration of example programs. Discussion on signed and unsigned conditional jump instructions with demonstration of example programs. Translating if…else code too assembly language.
Control Instructions – II:
A Recap of previous session. Translating high level repetition structure (for, while, ….) to its corresponding x86 assembly code using conditional jump instructions as well as using x86 loop instructions.
Lec-39.pdf
Lec-40.pdf
VL-39
VL-40
 
27GDB with PEDA Plugin:
A recap of gdb command line and text user interface mode. Downloading, installing, and configuring Python Exploit Development Assistance (PEDA) plugin to enhance the firepower of gdb. Debugging the x86-64 assembly programs using gdb with PEDA and a brief intro of using this plugin for reverse engineering and exploit development
Functions in Assembly Language – I:
What are functions? Why they are used in programming languages? Syntax of defining an assembly function in NASM and MASM. Understanding the working of x86-64 call and ret instruction. The usage of process run-time stack in function call. The concept and requirement of caller-saved and callee-saved registers
Lec-41.pdf
Lec-42.pdf
VL-41
VL-42
 
28Functions in Assembly Language – II:
Recap of functions in assembly language and use of call and ret instructions. How different operating systems allow x86-64 assembly programmers to pass and return values to and from functions. Designing an assembly function to display a single digit and a multi-digit decimal number on screen. Writing, assembling, linking and executing multi-file assembly programs on x86-64.
Function Calling Convention and FSF:
Recap of assembly language functions. Understanding function calling in high-level languages like C and C . The concept of Function Stack Frame (FSF) or Activation Record used to store data associated with a high-level function on the process run time stack. The x86-64 procedure prolog and procedure epilog for creating and removing FFSF from the stack. A demonstration of stack-based buffer overflow vulnerability and concept of exploiting it.
Lec-43.pdf
Lec-44.pdf
VL-43
VL-44
 
29Mixing C with x86-64 Assembly:
Recap of pushing and popping FSF to and from the process run time stack in high level languages. Calling C-Library functions from within an assembly program. Doing the reverse, i.e., Calling assembly functions from within a C program.
Getting User Input:
User input via system calls, library calls, and command line arguments. What are command line arguments and why we use them in high level languages as well as in assembly programming? Converting the string input received via command line to integer for further processing.
Lec-45.pdf
Lec-46.pdf
VL-45
VL-46
 
30Arrays:
Array address computation (pointer arithmetic) based on its types (bytes, words, double words and quad-words), General pattern for memory references, allocation arrays using malloc, processing arrays, filling with random numbers, printing array elements
String:
Understanding C-strings in assembly, x86 string instructions to store strings in memory, load strings from memory, comparing strings, and scanning strings for substrings
Lec-47.pdf
Lec-48.pdf
VL-47
VL-48
 
31Floating Point Instructions:
8087 floating point instructions that use stack of 80 bit floating point registers (ST0, ST1, …). Intel Core iseries floating point instructions that work with Streaming SIMD Extensions (SSE) 128-bit registers (xmm0, xmm1, …). The concept of Advanced Vector Extensions (AVX) that use 256-bit registers (ymm0, ymm1, …)
Data movement instructions (movss, movsd)
Arithmetic instructions (addss, addsd, subss, subsd, mulss, mulsd, divss, divsd)
Integer / floating point conversion instructions (cvtss2sd, cvtsd2ss, cvtss2si, cvtsi2ss, cvtsd2si, cvtsi2sd)
Floating point control instructions (ucomiss, ucomisd)
Floating point calling convention
Lec-49.pdf
VL-49
 
32Computer Performance and Parallel Processing Hardware:Lec-50.pdf
VL-50
 
 

Sr

Topic Description

Lecture Resources

Sessional Instruments

1

Overview of Machine Learning

VL-6.1

 

2

Math behind simple Linear Regression

VL-6.2

 

3

 

VL-6.3

 

4

 

VL-6.4

 

5

 

VL-6.5

 

6

 

VL-6.6

 

7

 

VL-6.7

 

8

 

VL-6.8

 

9

 

VL-6.9

 

10

 

VL-6.10

 

11

 

VL-6.11

 

12

 

VL-6.12

 

13

 

VL-6.13

 

14

 

VL-6.14

 

15

 

VL-6.15

 

16

 

VL-6.16

 

17

 

VL-6.17

 

18

 

VL-6.18

 

19

 

VL-6.19

 

20

 

VL-6.20

 

21

 

VL-6.21

 

22

 

VL-6.22

 

23

 

VL-6.23

 

24

 

VL-6.24

 

25

 

VL-6.25

 

26

 

VL-6.26

 

27

 

VL-6.27

 

28

 

VL-6.28

 

29

 

VL-6.29

 

30

 

VL-6.30

 

31

 

VL-6.31

 

32

 

VL-6.32

 

 

Scroll to Top