Preface

According to CHAOS Report, more than 50% of the software projects are failing. These projects are either canceled or delivered late or over budget. The reasons of failing projects may be technical or non-technical. One of the main technical reasons of failing projects is the low quality code. Because of the low quality code, maintaining the project and adding new features to the project is very hard. Also, the project may have a lot of bugs. These bugs may cause security vulnerabilities or data loss. So, writing a high quality code is very important for the success of a software project.

The Problem

High Quality Software

Writing a high quality code is not an easy task for developers and it requires a lot of effort and time. If a project has a long life cycle and its requirements are changing frequently, high quality code is a must not an option. For this reason, developers must strengthen their coding skills and learn new technologies in order to write high quality code.

Also, implementing a software project with a team is completely different than implementing a software project alone. In a team, developers need to follow some rules and conventions in order to work together effectively. Also, they need to use some tools and technologies to communicate with each other and to manage the project. For this reason, developers need to learn some concepts, technologies and tools in order to work in a team.

This book focuses on these challenges and its main goal is writing a good quality code in a team or a personal project. I personally do not believe that if you do not understand the underlying concepts and technologies, you can not write a good quality code. Yes, you can write a code that works but it will probably not be a high quality code. So, in this book, I will try to explain the underlying concepts and technologies in detail.

Structure of the Book

First of all, in order to learn these concepts and technologies, we will invent a problem and then we will try to solve it. Because, at its core software development is about solving problems. It is not about the problem itself. The problem always come from the real world. As developers, we do not need to explore the new problems. However we must understand the problem in order to solve it. Without understanding the problem, trying to solve it is just a waste of time.

After defining the problem, we will refresh our knowledge about the underlying concepts and technologies. Then, we will try to solve these problems in a right way. We will try to write a clean code that is easy to read and understand by following the best practices and conventions.

Some recommendations

The book covers a lot of topics and reading the book like a novel is not a good idea. Instead of that, you may read the some parts of the book again and again. For this reason, you can skip the chapters that you already know, but my advice is to read them because you may learn something new or notice something that you did not notice before.

It is very important to practice what you learn. So, while reading the book you should try to implement the examples in the book by yourself. I personally believe that, just reading the book will not be enough to grasp the whole book. You need to practice what you learn in order to understand the concepts deeply.

Also you can use the book as a reference. You can search for a topic in the book and then read the related chapter. You can

If you are in a trouble, please make sure that you have understood the concepts completely. If you are not sure, you can read the related chapter again. If you are still stuck, you can just google it. Also, you can use AI tools like ChatGPT to get help. However, please do not just copy and paste the code. Try to understand the code and then try to solve the challenge by yourself. Also, you can support the book with some online courses, YouTube videos, articles, books and any other resources. In each chapter you can find additional resources that you can use to learn the topic in detail.

Also, while writing your code, ask your self these questions:

• Is my code clean and readable?
• May i simplify the code or the solution?
• Is my code following the best practices and conventions?
• Is my code well-documented? (Comments, README etc.)
• Is my code tested? (Unit tests, integration tests etc.)
• Is my code secure?
• Is my code scalable?
• Is my code maintainable?
• Is my code consistent? (Naming conventions, code style etc.) (You can use linters and formatters to check this)

Assumptions and Requirements

First of all, in order to organize the content of the book, I need to know the audience of the book. I tried to write the book for the developers who are familiar with the programming concepts and technologies. However, I tried to explain the concepts in detail. So, if you are a beginner, you can still read the book. However, you may need to read some parts of the book again and again. For this reason, I have these assumptions about the audience of the book:

• You know a programming language. (You do not need to be an expert but you should know variables, conditional, loops, functions etc.)
• You know what an object oriented programming is. (You do not need to be an expert but you should know what a class, object, inheritance etc. is.)

Concepts and Technologies

Here are the concepts and technologies that we will be covered in the book:

• Command Line Interface Basics
• Server Side Rendering and Templating Engines
• Basic DOM Manipulation
• Client Side Rendering with React
• HTML & CSS
• Markdown Documentation
• Typescript and JavaScript
• Continuous Deployment (with GitHub Actions)
• Continuous Integration (with GitHub Actions)
• Cloud Computing (with AWS ECS, Fargate and Lambda)
• Cloud Databases (with AWS RDS, DynamoDB)
• Cloud Storage (with AWS S3)
• CDN (with AWS CloudFront)
• Authorization
• Authentication
• OAuth2, JWT and OpenID Connect
• Microservices Architecture
• RESTful API Design
• Clean Architecture / Hexagonal Architecture / Onion Architecture / Ports and Adapters
• Clean Code

Computer Basics

A computer is an automated machine that can perform tasks by following set of instructions. A computer may be a desktop, tablet, smartphone or a toaster. These computers may be an electronic device or a mechanical device. The idea behind the computers is not bound to the electronic devices.

A computer may work as an analog computer or a digital computer. Analog computers are the computers that work on the continuous data on the other hand digital computers work on the discrete data. Digital computers are the most common computers that we use in our daily life because they have an huge advantage over the analog computers when it comes to the accuracy and the precision.

Lets give an example; lets say that we have a computer that just adds two numbers. If we would use an analog fluid computer, the numbers would be represented by the pressure of the fluid. Therefore, the adding machine would create a pressure in its output that will be match the sum of the input pressures.

On the other hand, if we would use a digital computer, the numbers would be represented by the binary numbers. In order to represent binary numbers as a pressure, we need to pick constant pressure values for 0 and 1.

0s and 1s

In the digital world, everything is represented by the binary numbers. Binary numbers are the numbers that are represented by the 0s and 1s. The binary numbers are the most basic form of the data representation in the digital world. The binary numbers are used to represent the numbers, characters, images, sounds and videos.

You may think that how is it possible to represent everything with just 0s and 1s. This question is a good question and the answer is not trivial. Lets see;

Lets we want to store whether a light is on or off then we can represent the light is on with 1 and the light is off with 0. However, lets say that we want to store the color of the light. Lets assume that the light have 3 different colors then we can represent the state of the lisht bulb with 2 bits. Therefore, there are 4 different possible states;

As you can see we can concatenate the bits to represent more complex data. These groups of bits have a name;

However, generally people use hexadecimal numbers to represent the binary numbers because they have a more compact form. Hexadecimal numbers are the numbers that are represented by the 0-9 and A-F. Lets see the hexadecimal representation of the binary numbers;

To make the distinction between the hexadecimal and the decimal numbers, the hexadecimal numbers are generally prefixed with 0x. For example, the hexadecimal number 0x2A represents 8 bits of data in hexadecimal form. Also for the binaries the prefix 0b is used. For example, the binary number 0b110 represents 3 bits of data in binary form. If we are using decimals then we do not need to use any prefix.

Lets say that we want to store a number by using the binary numbers. Simply we can convert each number to its binary representation. For example, the number 5 is represented by 101 in binary.

However, this method to represent the numbers is not enough to represent the negative numbers. Therefore, we need to invent a new method to represent the negative numbers. The simplest method to represent the negative numbers is making the most significant bit as the sign bit (leftmost bit). If the sign bit is 1 then the number is negative, otherwise the number is positive. Lets look at some examples;

Logic Gates and Transistors

Transistors

CPU

Memory

Machine Code

I/O Devices

Summary

Exercises

Extra Resources

Computer Networks

OSI Layers

MAC

IP

UDP

TCP

DNS

Summary

Exercises

Extra Resources

Data Structures

Primitive Data Types

Linear Data Structures

Trees

Graphs

Hash Tables

Serialization Formats: JSON

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. You can use JSON to store your data or send your data over the network. JSON is a subset of JavaScript. It is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages. These properties make JSON an ideal data-interchange language.

Serialization Formats: XML

Serialization Formats: YAML

Summary

Exercises

Extra Resources

Programming Languages

Programming languages are used to describe the behaviot of a computer (includes mobile phones, tablets, embedded devices and servers) to perform a specific task. There are many programming languages and each of them has its own strengths and weaknesses. Some of the programming languages are general purpose and some of them are domain specific. Some of them are compiled and some of them are interpreted. Some of them are statically typed and some of them are dynamically typed.

A programming language is a formal specification of programs that can be executed by a computer. Every programming language has a syntax and a semantics.

The syntax of a programming language describes the valid "sentences" that can be written in the language. On the other hand, the semantics of a programming language describes the meaning of the "sentences" that can be written in the language.

Here are some langages that are popular in web development:

1. Javascript
2. Python
3. Java
4. C#
5. PHP
6. Go

Compiled and Interpreted Languages

Imperative Programming Language Paradigms

Procedural Programming

Object-Oriented Programming

Declarative Programming Language Paradigms

Functional Programming

Logic Programming

Types

Linters

Formatters

Package Managers

What is package manager? Package manager constraints Lock files Workspaces Monorepos

Code Editors

Using VSCode

You can use VSCode as your IDE. It is a good IDE for Javascript and Node.js development. Vscode has a lot of plugins that you can use. For instance, you can use Prettier plugin to format your code. You can use ESLint plugin to check your code quality. You can use GitLens plugin to see git history.

Visual studio supports sharing your IDE settings across different machines. You can use .vscode folder to store your settings. You can put your settings in settings.json file. You can put your tasks in tasks.json file. These are files are very imprtant because working in projects with different settings can be a problem. So, you need to keep your settings in version control. (Described in the next section)

Markdown

You will write your documentation in markdown format. Markdown is a lightweight markup language with plain text formatting syntax. It is designed so that it can be converted to HTML and many other formats using a tool. Markdown is often used to format README files, for writing messages in online discussion forums, and to create rich text using a plain text editor. You can check markdown format from here.

Popular Programming Languages

Summary

Exercises

Extra Resources

Software Architecture

Clean Code

One of the objectives of the all challanges is to write clean code. Clean code is a code that is easy to read, understand and change. In order to write clean code, you need to follow some rules. These rules explain in detail in challanges.

• Writing a complicated code is not a good thing. Try to write simple code. (Good comments, small functions, no over abstraction etc.)

Styling and Formatting

A code can be seen unclear if it is not formatted correctly. Formatting covers spaces, tabs, new lines, semicolons etc. A code formatter is a tool that automatically formats your code. Formatting means changing the code style. For instance, you can use a code formatter to change

• all double quotes to single quotes
• all tabs to spaces
• all semicolons to no semicolons

or any other code style. A code formatter is a tool that you can use to format your code. It is a good practice to use a code formatter. It will help you to keep your code clean. However, please do not forget that you need to setup your code formatter correctly. Also, different code formatting rules across different projects can be a problem. So, you need to add a configuration file to your project in order to keep your code style consistent. Also, configuration file should be under version control. So, all developers can use the same code style.

Naming

Naming: You should be careful about naming. You should use meaningful names for your variables, functions, classes etc.

Comments

Your comments should be as short as possible. You should write comments to explain your code but in general your code should be self-explanatory. You should not write comments that are not needed. You can explain algorithms, data structures, design patterns etc. in your comments. You should not write comments that are explaining what your code does. Instead, your code should explain what it does in a clear way.

Nesting

You should avoid deeply nested code. You should use early returns to avoid nested code.

Immutablity

One of the most bug-prone parts of a code is mutable variables (or variables). Because, you can change the value of a variable at any time from any place. This can cause a lot of bugs. In order to avoid these bugs, you should use immutable variables as much as possible. Instead of using let and var, you should use const in order to make your variables immutable. Also, you should avoid mutating objects and arrays. You should use map, filter, reduce functions to manipulate arrays.

Also you should be careful about scoping. Because scopes can restrict the usage of a variable. If you are defining a variable in a global scope, you can use it from anywhere. This can cause a lot of bugs. For this reason, please try to use local variables as much as possible and keep your scopes as small as possible.

It may be seem to be impossible to avoid mutable variables. However, you can use some techniques to avoid them. For instance, you can use Map, Filter and Reduce functions can be used. They are higher order functions which means that they take a function as an argument and they return a new array. These functions are very useful and they are reduces the need for loops and mutable variables. You should use these functions as much as possible. Also there are some other functions in javascript that you can use to manipulate arrays. For instance, forEach, some, every, find, findIndex etc. You should use these functions as much as possible. Also, you can use lodash library to manipulate objects and arrays.

Also, you should return results of a function with return statement. Mutating a global variable and then reading it from another function is a bad practice. Instead of that, you should return the result of a function with return statement. This will make your code much more clear and easy to understand.

If you want to learn more about immutability, you can check Functional Programming Paradigm.

Pure Functions

There is a difference between functions in mathematics and functions in programming. In mathematics, a function is a relation between a set of inputs and a set of possible outputs. Therefore, a function satisifies two properties:

1. A function should return the same output for the same input.
2. A function should not have side effects.

However, in programming, a function can have side effects and it can return different outputs for the same input. For instance, a random number generator function is a function that returns different outputs for the same input. It does not take any arguments but it returns a different output each time you call it. (If it would be a function in a mathematical sense, it should return only one value because it does not take any arguments.) Also, a function that prints something to the screen is a function that has side effects. It does not return anything but it changes the state of the screen. The functions that satisfy these two properties are called pure functions.

In some functional programming languages, all functions must be pure functions (such as Haskell). However, in JavaScript, you can use pure functions as much as possible. Pure functions are very useful because they are easy to test. You can test a pure function by giving it an input and checking the output. If the output is the same as you expected, then the function is working correctly. Also, pure functions are easy to understand and most importantly, they are easy to compose. Therefore, building complex systems with pure functions is much easier than building them with impure functions. Because, they are garantueed to work correctly for each individual function and composing them will not result any unexpected behavior.

Map, Filter, Reduce

Extra Resources

https://youtu.be/CFRhGnuXG-4?si=dOHjdiMcc5xxgEuM

Version Control

Writing code is a complex and continuous process. For this reason, your code will change over time. In order to keep track of these changes, you will need a tool manage the versions of your code. This is where version control systems come into play. A version control system is a tool that helps you to manage the changes in your code. It allows you to keep track of the changes that you made in your code. Also, it allows you to revert your code to a previous state if something goes wrong. In addition to these, it allows you to collaborate with other developers. You can work on the same codebase with other developers without any conflicts. Also, you can see who did what in the codebase. This is very useful when you are working with a team.

There are many version control systems in the industry. Some of them are centralized and some of them are distributed. Centralized version control systems have a central server that stores the codebase. Developers can clone the codebase from the server and they can push their changes to the server. However, this approach has some drawbacks. For instance, if the server goes down, then no one can access the codebase. Also, if the server is slow, then the developers will have to wait for the server to respond. In addition to these, if the server is hacked, then the codebase can be stolen. These are some of the drawbacks of centralized version control systems.

Distributed version control systems are more popular in the industry. In distributed version control systems, every developer has a copy of the codebase on their local machine. This is very useful because you can work offline and you can commit your changes to your local repository. Also, you can push your changes to the remote repository when you are online. This approach is more secure and faster than centralized version control systems. Also, it is more reliable because you have a copy of the codebase on your local machine.

Git

Git is a version control system that is widely used in the industry. This book especially focuses on Git because of its popularity. Git is a distributed version control system which means that every developer has a copy of the repository on their local machine. This is very useful because you can work offline and you can commit your changes to your local repository. Also, you can push your changes to the remote repository when you are online.

Everything in git starts with a repository. A repository is a place where you store all of the changes and versions of your project. A repository can be located on your local machine and/or on a remote server. You can create a repository by running git init command in your project directory. This command will make the current directory a git repository. When you run the command, you will notice that it will not do any changes other than creating a .git folder in your project directory. Therefore, git stores everything in .git folder. This folder is a hidden folder and you should not directly modify it if you do not really know what you are doing.

After creating a repository, you can start tracking your files in your project directory. The directory that you are using for your project is called working directory in git. Working directory is completely separate from the repository. Git itself may create, modify or delete files in the working directory according to the commands that you run. Also some commands may affect your repository. Therefore we have two different type of data storage behind the scenes. One is the working directory and the other is the repository.

The synchronization between the working directory and the repository is one of the main concepts in git. When you make changes in your working directory, you should stage them before committing. Staging is the process of preparing your changes to be committed. A commit is a snapshot of your changes. It is like a save point in a game. We can add a file to the staging area by running git add <file> command. After collecting all the changes that we want to commit, we can run git commit -m "Commit message" command to commit the changes. The -m flag is used to add a message to the commit. This message should describe what you did in this commit. It is very important to write meaningful commit messages.

We can see the current status of the repository and the changes that we made by running git status command. This command will show us the files that are modified, staged or not staged. Also, we can see the history of the repository by running git log command.

# Create a directory
mkdir git-done-right
cd git-done-right

# Initialize a git repository
git init

# Create a file (you may create your files first and then run the init command after that)
echo "Hello, World!" > hello.txt

# Add the file to the staging area (Which means that we want to include all the changes in this file to the next commit)
git add hello.txt

# Lets check the status of the repository
git status

# Commit the changes
git commit -m "Initial commit."

# Check the status again
git status

# Make some changes
echo "\n" >> hello.txt
echo "How are you?" >> hello.txt

# Check the status again
git status

# Add the changes to the staging area
git add hello.txt

# Commit the changes
git commit -m "Added a new line."

# See all the commits
git log

We can imagine our changes in the repository like a chain. Every commit is a link in this chain. We can go back to a previous commit by running git checkout <commit-hash> command. The commit-hash is the hash of the commit that you want to go back. You can see the commit hashes by running git log command. After going back to a previous commit, you can go back to the latest commit by running git checkout master command. The master is the name of the branch that we are currently working on. We will talk about branches later. git checkout command modifies the working directory according to the commit that you specified. Therefore, you should be careful when you are using this command. If you have uncommitted changes in your working directory, then you may lose them.

We have two commits right now and our commit history is looks like this:

gitGraph
  commit id:"A"
  commit id:"B"

Lets imagine that we want to do some experiments with our code and we will decide later whether we will keep them or not. In this case, we may utilize branches. A branch is a pointer to a specific commit. At this point, this may be very confusing. For this reason, we should not forget that each commit has a parent commit which refers to the previous commit.

flowchart RL
  A-- parent -->NULL;
  B-- parent -->A;

However, instead of drawing the parent pointer explicitly, like the above, we will use git diagrams to represent the branches. In this case, you should keep in mind that each commit is represented by a circle and the parent commit is represented is the commit that is located on the left side of the commit and connected to the commit with an edge.

When we create a branch, git will create a new pointer to the commit that we are currently on. This pointer is called a branch. We can create a branch by running git branch <branch-name> command. After creating a branch, we can switch to the branch by running git checkout <branch-name> command. This command will switch to the branch that you specified.

gitGraph
  commit id:"A"
  commit id:"B"
  branch experiment-1
  commit id:"C"
  checkout main
  branch experiment-2
  commit id:"D"
  checkout main
  merge experiment-2

At this point, our repository is completely stored on our local machine. If we want to share our code with others, we should store them in a place where others can access it. This place is called a remote repository. A remote repository is a repository that is stored on a server which can be accessed by other developers. The server may be publicly accessible or it may be a private server.

In order to work with remote repositories, the first step is to introduce the remote repository to our local repository. We can do this by running git remote add origin <remote-url> command. This command will add a remote repository to our local repository. We may define multiple remote repositories for our local repository and for this reason, we are providing a name for the remote repository. This name is called an alias. In this case, we are using origin as an alias. This is a common convention in the industry. You can use any name that you want. After adding the remote repository, we can push our changes to the remote repository by running git push origin master command. This command will push our changes to the remote repository. The master is the name of the branch that we are pushing. We will talk about it later.

# Add remote repository
git remote add origin https://github.com/cebecifaruk/git-done-right

# Push the changes to the remote repository
git push origin master

The push command simply sends all the commits that are not in the remote repository to the remote repository. Therefore, in the remote repository, we have all the knowledge that is required to build the current working directory of the project. This means that we can also get all the commits from a different machine or in a different directory. In order to "download" a project from a remote repository, we should take these steps:

# Create a directory
mkdir git-done-right-clone

# Change the directory
cd git-done-right-clone

# Initialize a git repository
git init

# Add the remote repository
git remote add origin

# Pull the changes from the remote repository
git pull origin master

However, instead of doing all the steps manually, git provides a command to do all these steps in one command. This command is git clone <remote-url>. This command will take all the steps that we mentioned above and it will create a new directory with the name of the repository.

Git Behind the Scenes

Github

Github is a web-based platform that is built on top of git where you can store your repositories and collaborate with other developers. It is a very popular platform in the industry. It provides many features that are not available in git itself.

You can use Github personally by creating a personal account or you can use it as an organization by creating an account for your organization. However, instead of creating an account for an organization, you can create an organization in Github. An organization is just a group of users at the end of the day. You can add users to your organization and you can manage their permissions. Also, you can create teams in your organization. A team is another group of users that are in the organization.

In addition to these, Github is like a social media platform for developers. You can follow other developers, star their repositories and watch their repositories. Starring a repository means that you like the repository. Watching a repository means that you want to get notifications about the repository. You can also fork a repository. Forking a repository means that you are creating a copy of the repository in your account. You can make changes in the forked repository and you can create a pull request to the original repository. A pull request is a request to merge your changes to the original repository. The owner of the original repository can accept or reject your pull request.

In Github, you can track the tasks that you need to do, and these are called issues. Issues may be a bug, a feature request, a question etc. You can assign issues to other developers and yourself. Issues are organized per repository. If you are working on multiple repositories, you can use projects to organize your issues. Projects are like a kanban board where you can organize your issues in columns.

Also, as you can store your documentation in the repository, you can also store your documentation in repository's wiki. A wiki is a place where you can store your documentation in markdown format. You can create pages, link them to each other and you can organize them in a tree structure.

Monorepo vs Polyrepo

Best Practices

Your commits should describe what you did but it should not consists of only changes. For instance README file updated or x file changed messages are not good commit messages. Instead, you can use Added a new section about x for README file updated or Added a new function to do y for x file changed. Therefore, it should give an idea about what you did. It is not needed to list all crud operations that you did because they are already listed in the commit itself.

Also please be consistent between your commit messages. For instance, if you use imperative mood for your commit messages, you should use it for all of your commit messages. If you use past tense, you should use it for all of your commit messages. Also syntacticly they should be consistent. For instance, you can start with a big letter and you can put a dot at the end of the message. But you should be consistent between your commit messages.

It is very important to keep git history clean. You should not commit files that are not related to your project. For instance, you should not commit your node_modules folder, IDE files, temporary files etc. You should only commit files that are related to your project. Also, git stores your history. So, you should not grow your git repository size by committing unnecessary files. PLease be careful about binary files. Git is not good at storing binary files. So, you should not commit binary files (If you really need them, then you can). But please be careful about them.

Another thing is using .gitignore file to ignore files that you don't want to track. For instance, you don't need to track node_modules folder. You can ignore it by adding node_modules to .gitignore file. However, instead of writing it manually, you can use gitignore.io to generate .gitignore file for your project.

Summary

Exercises

Extra Resources

Operating Systems

What is Kernel?

What is a program?

Command Line Interface

There are 2 common approaches to interact with a computer. The first one is GUI (Graphical User Interface) and the second one is CLI (Command Line Interface). GUI is a visual interface that you can use to interact with your computer. Command line interface is a text-based interface that you can use to interact with your computer. In CLI, you are giving a command to your computer and the computer runs it and gives a result to you by printing it to the screen. There are a lot of commands that you can use in CLI. Here are some of them:

A cli command is nothing more than a program that you can run from your terminal or shell. Each cli command is an executable file located in your PATH. When you run a command, your shell will search for the executable file in your PATH and if it finds it, it will run it. So, you need to put your executable file in your PATH in order to run it from anywhere. You can check your PATH by using echo $PATH command. It is a list of directories separated by :. You can put your executable file in one of these directories.

Let's look at how ls command works. When you write ls in your terminal, it will try to find an executable file named ls in your PATH. It will check each directory in your PATH one by one. If it finds it in the first directory, it will run it directly. If it does not find it in the first directory, it will check the second directory and so on. If it does not find it in any of these directories, it will give an error. You can check where ls command is located by using which ls command. It will give you the path of the executable file. Also, terminal will check the file permissions. If the file is not executable, it will give an error. For this reason you need to know the concept of users, user groups and file permissions.

There are two types of executable files. The first one is a binary ELF file. The second one is a script file. A script file is a text file that contains shebang at the beginning of the file. Shebang is a special comment that starts with #!. It tells the shell which program to use to run the script. For instance, if you write #!/bin/bash at the beginning of your script file, shell will use bash to run your script. If you write #!/usr/bin/env node, terminal will run env command with node as the first argument. And as a second argument, it will give your script file's path. So, env command will run node with your script file's path as the first argument. Please use shebang at the beginning of your script file. Otherwise, your script will not run.

In the challenge, you will implement an argument parser in order to parse the arguments that are given to your program. You can get these arguments by using process.argv array. It is an array that contains the arguments that are given to your program. You need to follow these subcommands in order to implement your TODO app:

1. list: mytodo list also mytodo (default subcommand) (--done and --undone flags should be supported)
2. add: mytodo add "A sample task description"
3. done: mytodo done 1
4. undone: mytodo undone 1
5. delete: mytodo delete 1
6. update: mytodo update 1 "A new task description"

Standard IO, Exit Code, Environment Variables and Arguments

Each program ran from the terminal has an exit code. This exit code is just a number between -128 and 127. It is used to indicate the status of the program. If the program runs successfully, it will return 0. If it fails, it will return a non-zero number (You can use any non zero number between -128 and 127). You can check the exit code of the last ran program by using echo $? command. It will print the exit code of the last ran program. Exit codes are very important to check the status of a command.

Also, each program has 3 standard streams in order to interact with the user. A stream is a just a file that you can write to or read from. There are 3 standard streams in a program. These are stdin, stdout and stderr. stdin is the standard input stream. It is used to read input from the user (keyboard). stdout is the standard output stream. It is used to write output to the user (screen). stderr is the standard error stream. It is used to write error messages to the user (screen).

However, also a command can be started with some arguments. These arguments are called command line arguments. Arguments are just an array of strings. And your arguments should be separated by a space. For instance, if you write ls -l -a in your terminal, ls command will be started with 3 arguments. The first argument will be ls, the second argument will be -l and the third argument will be -a.

Basic Commands

Users and Groups

File System

Permissions

Processes

Package Managers

Shell Scripting

Commands

Environment Variables

Also, there is a concept of environment variables. Environment variables are just key-value pairs that are stored in your shell. These variables are used to get some common configuration values across different programs. For instance, you can use EDITOR environment variable to get the editor that you want to use. You can use PATH environment variable to get the directories that you want to search for executable files. Do you see the pattern? Your shell uses PATH environment variable to search for executable files. You can use process.env object to access these environment variables in your program. In this challenge, you will use TODO_USERNAME environment variable to get the assignee of the TODOs.

Stdin, Stdout, Stderr

Exercises

Internet

Hyper Text Markup Language (HTML)

A markup language is a language that is used to annotate text. The reason why it is called markup language is that you are marking up the text. Why we need to annotate text? Lets sey that we want to write a book. We need to add some formatting to the text. For instance, we may need to make some words bold, italic or underline. Also we may need to add some images, tables, lists etc. to the text. However, at the and of the day, we need to store the text in a text file. So, we need a way to annotate the text. This is where markup languages come into play. There are a lot of markup languages. For instance:

• Markdown
• Latex
• HTML
• XML

HTML is different from other markup languages. Because you can view the result in a web browser. It is used to create web pages. A web page is a text file that contains HTML tags. A tag is a special text that starts with < and ends with >. There are two types of tags. The first one is a self closing tag. It does not have a closing tag. For instance, <br> is a self closing tag. The second one is a normal tag. It has a closing tag. For instance, <p> is a normal tag and </p> is its closing tag.

Historically, HTML was used to create static, not styled web pages. If you want to style your web page, you need to use CSS. CSS is a language that is used to style HTML elements. However, in order to create dynamic web pages, you need to use a programming language. This is where javascript comes into play. Javascript is a programming language that is used to create dynamic web pages.

However, also you can serve dynamic web pages with a trick called server side rendering. This means that you can create a web page with a programming language and send it to the client. This is what you will do in this challange.

Cascading Style Sheets (CSS)

Hyper Text Transfer Protocol (HTTP)

In order to serve your webpage, you need a some kind of protocol. This protocol is called HTTP. HTTP is a protocol that is used to transfer hypertext. Hypertext is a text that contains links to other texts. For instance, a web page is a hypertext. It contains links to other web pages.

HTTP is a request-response protocol. This means that a client sends a request to the server and the server sends a response to the client. A request contains a method, a path and list of headers. A response contains a status code, some headers and a body. A method is a verb that tells the server what to do. For instance, GET method is used to get a resource from the server. A path is a string that tells the server which resource to get. For instance, / path is used to get the root resource. A header is a key-value pair that contains some information about the request. For instance, Content-Type: text/html header tells the server that the body of the request is a HTML document. A status code is a number that tells the client what happened. For instance, 200 status code means that everything is OK. A body is a string that contains the response. For instance, if the response is a HTML document, the body will contain the HTML document.

Sample HTTP request:

GET / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.64.1
Accept: */*

As you can see the first line contains the method (GET), the path (/) and the HTTP version (HTTP/1.1) separated by spaces. The first line is called request line and the other lines are called headers. These lines are separated by \r\n. After sending all headers, in order to tell the server that the headers are finished, you need to send an empty line. After that, you can send the body of the request. In this case, the body is empty.

Sample HTTP response:

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: text/html; charset=utf-8
Content-Length: 25

<h1>Hello World</h1>

As you can see the first line contains the HTTP version (HTTP/1.1), the status code (200) and the status message (OK) separated by spaces. The first line is called status line and the other lines are called headers. These lines are separated by \r\n. After sending all headers, in order to tell the client that the headers are finished, you need to send an empty line. After that, you can send the body of the response. In this case, the body is <h1>Hello World</h1> which is an HTML document.

TLS/SSL

API

REST API

REST (Representational State Transfer) is an architectural style for developing web services. REST is popular due to its simplicity and the fact that it builds upon existing systems and features of the internet's HTTP in order to achieve its objectives, as opposed to creating new standards, frameworks and technologies. REST is a stateless client-server architecture where web services

Javascript

Browsers use JavaScript to make web pages dynamic and interactive. JavaScript allows to manipulate the content, structure, and style of a web page in real-time, respond to user inputs, make asynchronous requests to the server, and much more. We are going to cover manipulating the web-content in this challenge. The JavaScript engine in the browser parses the JavaScript code. (V8 Engine for Chrome, SpiderMonkey for Firefox) The parsed JavaScript code is converted into machine code and executed. The JavaScript code can manipulate the DOM, making real-time changes to the web page content, structure, or style. JavaScript is single-threaded, but it can handle asynchronous operations using the event loop and callback queue. When an asynchronous operation is encountered, it’s offloaded, and its callback is placed in the callback queue. The event loop continuously checks if the main thread (where the synchronous code runs) is free. If it is, the event loop takes the next callback from the callback queue and executes it.

DOM

The DOM is a programming interface for HTML and XML documents. It represents the structure of a document as a tree of objects. Each node in this tree corresponds to a part of the document, for example, an element, an attribute, or text content. Mechanism that browsers use to convert HTML into DOM is below:

• The browser parses the HTML content line by line.
• It converts HTML elements, attributes, and text into objects.
• These objects are structured in a tree-like form, mirroring the nested structure of the HTML tags.
• Each object represents a part of the page and contains properties and methods that you can manipulate using JavaScript.

Resource about DOM

Modules

Writing client side rendered single page applications is not an easy task and it is not practical to write everything in one file. For this reason, you should split your code into multiple pieces. For this reason, we are using modules. A module is a piece of code that hides or exposes some functionality, and most importantly, it is reusable. Therefore, it is a collection of functions, variables, and classes that are bundled together. These functions, variables, and classes are not accessible from outside the module unless they are explicitly exported. In this way, you can hide the internal implementation details of a module and expose only the necessary parts. This is called encapsulation. Historically, there are several ways to define modules in JavaScript: CommonJS, AMD, UMD, and ES Modules. Lets examine them one by one.

In CommonJs modules, your script work in a context which has a predifined set of variables. You can use require function to import a module and module.exports object to export a module. For instance, if you want to use a library called lodash, you can import it like this:

const _ = require('lodash');

require function is a synchronous function that loads the module if it is not loaded before and returns the module object. If you want to export a function from a module, you can use module.exports object like this:

module.exports = function () {
  console.log('Hello World');
};

Here is an example of a CommonJS module:

// math.js
function add(a, b) {
  return a + b;
}

module.exports = {
  add,
};

// index.js
const math = require('./math');

console.log(math.add(1, 2));

CommonJS modules are synchronous. When you import a module, it is loaded immediately. This is not a problem in server side applications. However, in client side applications, it is a problem because you have to load all the modules before the application starts. For this reason, CommonJS modules are used by Node.js, not in the browser.

Fetch and XMLHttpRequest API

There are multiple ways to send HTTP requests from browser. The first one is to use fetch API. This is a native API and it is supported by all browsers. The second one is to use XMLHttpRequest API. This is an old API and it is supported by all browsers. The third one is using a third party library like axios. This is a library that you can use to send HTTP requests from browser. These libraries actually wrap XMLHttpRequest or fetch API. Third party libraries generally provide easier to use APIs. However, they are not native APIs. This means that you need to include them in your project. Also, they increase the size of your project. It is recommended to use axios API. However, if you try to use fetch and XMLHttpRequest API, it will help you to understand how HTTP requests work under the hood.

Bundlers

In a typical web application, you can use javascript sources with <script src=".."> tags. However, if you want to use libraries in your app, then you should import every library with <script src=".."> tags. Lets say you want use

CSR, SSR, SSG

Client side rendering is a technique that you render your HTML in browser. This means that you don't need to render your HTML in your server. This means that you don't need to use templating engines like EJS or handlebars.

Summary

Extra Resources

• HTML Wikipedia Page
• HTML in 100 Seconds
• CSS in 100 Seconds
• HTML in 5 minutes
• HTML Tutorial for Beginners: HTML Crash Course

Exercises

Database

A database is an organized collection of data, generally stored and accessed electronically from a computer system. According to this definition, our file-based implementation is also a database.

Types of Databases

Query Language

A query language is a domain specific language used to make queries in databases and information systems (In this context context is a database). Complex queries can be expressed in a query language. There are many types of query languages.

SQL

SQL is the standard language for relational database management systems. SQL statements are used to perform tasks such as update data on a database, or retrieve data from a database.

In SQL, data is stored in tables. A table is a collection of related data entries and it consists of columns and rows.

SQL Injection Attack

SQL injection is a malicious technique in computer engineering where an attacker manipulates input data to insert or execute arbitrary SQL query within a web application's database. This vulnerability occurs when user inputs are not properly validated or sanitized, allowing unauthorized access to the database and potentially exposing sensitive data. SQL injection poses a significant security risk, as it can lead to data breaches, unauthorized data manipulation, and even full system compromise if not adequately mitigated. It is not only made by attackers but can occur by mistake by filling forms that runs SQL queries behind.

Command Line Interface Challenge

Each software project has a structure and each of them starts with a scaffolding. Scaffolding is a process of creating the initial structure of the project. I will not use any project scaffolding tools in this book and also I do not recommend you to use them. Because, they will hide the underlying concepts and technologies from you. Instead of that, I will create the initial structure of the project by hand and I will explain each step in detail. So, you can understand the underlying concepts and technologies in detail.

# Create a new directory for our project
mkdir tdr

# Change the directory to the project directory
cd tdr

# Initialize the git repository
git init

# Initialize the .gitignore file
curl https://www.toptal.com/developers/gitignore/api/linux,macos,windows,node > .gitignore

# Initialize the package.json file
npm init -y

# Create the src directory
mkdir src
touch src/index.ts

# Create a directory for Github Actions
mkdir .github

# Create a directory for VSCode settings
mkdir .vscode

# Initialize the code formatter
npm i -D prettier
touch .prettierrc

# Initialize the linter
npm i -D eslint
touch .eslintrc

# Install the typescript and tsx (Typescript execute)
npm i -D typescript tsx @types/node

# Initialize the typescript configuration
npx tsc --init

# Create a README.md file
touch README.md

Lets add some scripts to the package.json file.

  "main": "src/index.ts",
  "scripts": {
    "format": "prettier --write .",
    "format:check": "prettier --check .",
    "lint": "eslint .",
    "start": "tsx src/index.ts",
    "dev": "tsx --watch src/index.ts",
  },

Implementing without any structure

Although we have a lot of requirements, we will start with a simple subset of them. For this reason, we will just attach title, description, and done fields to out todo list. Also we will just implement basic BREAD operations such as list, add, done, undone, delete (notice that we do not have generic edit or update operation in addition to single read). We will implement all BREAD operations later.

According to these requirements, we can store all of our todos in a single JSON file and the structure of the file might be like this:

[
  {
    "id": 1,
    "title": "A sample task description",
    "description": "This is a sample task description",
    "done": false
  }
]

The plain is simply like this:

1. Get command line arguments.
2. Check the existence of the todo list file. If it does not exist, create it.
3. Parse the command line arguments and decide which operation to perform.
4. Read the todo list file and parse it.
5. Perform the operation.
6. Write the updated list back to the file. (If it is necessary)

Lets implement the first version of our todo list application:

// File: src/index.ts
import process from 'process';
import fs from 'fs';

// Define what a todo looks like
interface Todo {
  id: string;
  title: string;
  description: string;
  completed: boolean;
}

// process.argv is provided by Node.js and it is an array that contains the command line arguments
// The first element is the path to the Node.js executable
// The second element is the path to the script file
// The third element is the subcommand
// The rest of the elements are the arguments
// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;

// If the todo list file does not exist, create it
if (!fs.existsSync('todos.json')) {
  fs.writeFileSync('todos.json', '[]');
}

switch (subcommand) {
  case 'add': {
    const [title, description] = args;

    // Read todo list file and parse it
    const fileConent = fs.readFileSync('todos.json', 'utf-8');
    const todoList = JSON.parse(fileConent) as Todo[];

    // Generate a new todo
    const id = Math.random().toString(36).substr(2, 5);
    const newTodo: Todo = {
      id: id,
      title,
      description,
      completed: false,
    };

    // Add the new todo to the list
    todoList.push(newTodo);

    // Write the updated list back to the file
    const updatedFileContent = JSON.stringify(todoList, null, 2);
    fs.writeFileSync('todos.json', updatedFileContent);

    console.log('New todo added');
    break;
  }
  case 'list': {
    // Read todo list file and parse it
    const fileConent = fs.readFileSync('todos.json', 'utf-8');
    const todoList = JSON.parse(fileConent) as Todo[];

    // Print the list of todos
    for (const todo of todoList) {
      const status = todo.completed ? 'X' : ' ';
      console.log(`- [${status}] (id: ${todo.id}) ${todo.title}`);
      if (todo.description) console.log(`\t${todo.description}`);
    }

    break;
  }
  case 'done': {
    const [id] = args;

    // Read todo list file and parse it
    const fileConent = fs.readFileSync('todos.json', 'utf-8');
    const todoList = JSON.parse(fileConent) as Todo[];

    // Find the todo with the given id
    const todo = todoList.find((todo) => todo.id === args[0]);
    if (!todo) {
      console.log('Todo not found');
      process.exit(1);
    }

    // Mark the todo as completed
    todo.completed = true;

    // Write the updated list back to the file
    const updatedFileContent = JSON.stringify(todoList, null, 2);
    fs.writeFileSync('todos.json', updatedFileContent);

    // Print the message
    console.log('Todo marked as done');

    break;
  }
  case 'undone': {
    const [id] = args;

    // Read todo list file and parse it
    const fileConent = fs.readFileSync('todos.json', 'utf-8');
    const todoList = JSON.parse(fileConent) as Todo[];

    // Find the todo with the given id
    const todo = todoList.find((todo) => todo.id === args[0]);
    if (!todo) {
      console.log('Todo not found');
      process.exit(1);
    }

    // Mark the todo as not completed
    todo.completed = false;

    // Write the updated list back to the file
    const updatedFileContent = JSON.stringify(todoList, null, 2);
    fs.writeFileSync('todos.json', updatedFileContent);

    // Print the message
    console.log('Todo marked as undone');

    break;
  }
  case 'delete': {
    const [id] = args;

    // Read todo list file and parse it
    const fileConent = fs.readFileSync('todos.json', 'utf-8');
    const todoList = JSON.parse(fileConent) as Todo[];

    // Delete the todo with the given id
    const index = todoList.findIndex((todo) => todo.id === args[0]);
    if (index === -1) {
      console.log('Todo not found');
      process.exit(1);
    }
    todoList.splice(index, 1);

    // Write the updated list back to the file
    const updatedFileContent = JSON.stringify(todoList, null, 2);
    fs.writeFileSync('todos.json', updatedFileContent);

    // Print the message
    console.log('Todo deleted');

    break;
  }
  default:
    // Print help messages
    console.log(`Unknown subcommand`);
    console.log(`Usage: tdr <subcommand> [args]`);
    console.log(`Subcommands: add, list, done, undone, delete`);

    // Exit with an error code
    process.exit(1);
}

And we can test the application by running the following commands:

# Add a new todo
npm run start add "A sample task description" "This is a sample task description"
npm run start add "Another task" "This is another task"

# List the todos
npm run start list

# Mark a todo as done
npm run start done #PutYourTodoIdHere#

# Mark a todo as undone
npm run start undone #PutYourTodoIdHere#

# Delete a todo
npm run start delete #PutYourTodoIdHere#

Right now, we have a "working" todo list application but this is not the best implementation. Lets improve it.

Implementing with functions

In the previous implementation, we did not utilize functions. Without changing the requirements and the file structure of the todo list, lets refactor the code to use functions.

First of all, we will use two different files: index.ts and todo.ts. We will move the todo related functions to the todo.ts file and we will import them in the index.ts file.

// File: src/index.ts
import process from 'process';
import * as todo from './todo';

// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;

switch (subcommand) {
  case 'add': {
    const [title, description] = args;
    todo.addTodo(title, description);
  }
  case 'list': {
    todo.listTodos();
    break;
  }
  case 'done': {
    const [id] = args;
    todo.markTodoAsDone(id);
    break;
  }
  case 'undone': {
    const [id] = args;
    todo.markTodoAsUndone(id);
    break;
  }
  case 'delete': {
    const [id] = args;
    todo.deleteTodo(id);
    break;
  }
  default:
    // Print help messages
    console.log(`Unknown subcommand`);
    console.log(`Usage: tdr <subcommand> [args]`);
    console.log(`Subcommands: add, list, done, undone, delete`);

    // Exit with an error code
    process.exit(1);
}

// File: src/todo.ts
import fs from 'fs';

// Define what a todo looks like
export interface Todo {
  id: string;
  title: string;
  description: string;
  completed: boolean;
}

// Read todo list file and parse it
function readTodos(): Todo[] {
  if (!fs.existsSync('todos.json')) {
    fs.writeFileSync('todos.json', '[]');
  }

  const fileConent = fs.readFileSync('todos.json', 'utf-8');
  const todoList = JSON.parse(fileConent) as Todo[];

  return todoList;
}

// Write the updated list back to the file
function writeTodos(todoList: Todo[]): void {
  const updatedFileContent = JSON.stringify(todoList, null, 2);
  fs.writeFileSync('todos.json', updatedFileContent);
}

export function addTodo(title: string, description: string): void {
  const todoList = readTodos();

  // Generate a new todo
  const id = Math.random().toString(36).substr(2, 5);
  const newTodo: Todo = {
    id: id,
    title,
    description,
    completed: false,
  };

  // Add the new todo to the list
  todoList.push(newTodo);

  writeTodos(todoList);

  console.log('New todo added');
}

export function listTodos() {
  const todoList = readTodos();

  // Print the list of todos
  for (const todo of todoList) {
    const status = todo.completed ? 'X' : ' ';
    console.log(`- [${status}] (id: ${todo.id}) ${todo.title}`);
    if (todo.description) console.log(`\t${todo.description}`);
  }
}

export function markTodoAsDone(id: string) {
  const todoList = readTodos();

  // Find the todo with the given id
  const todo = todoList.find((todo) => todo.id === id);
  if (!todo) {
    console.log('Todo not found');
    process.exit(1);
  }

  // Mark the todo as completed
  todo.completed = true;

  writeTodos(todoList);

  // Print the message
  console.log('Todo marked as done');
}

export function markTodoAsUndone(id: string) {
  const todoList = readTodos();

  // Find the todo with the given id
  const todo = todoList.find((todo) => todo.id === id);
  if (!todo) {
    console.log('Todo not found');
    process.exit(1);
  }

  // Mark the todo as not completed
  todo.completed = false;

  writeTodos(todoList);

  // Print the message
  console.log('Todo marked as undone');
}

export function deleteTodo(id: string) {
  const todoList = readTodos();

  // Delete the todo with the given id
  const index = todoList.findIndex((todo) => todo.id === id);
  if (index === -1) {
    console.log('Todo not found');
    process.exit(1);
  }
  todoList.splice(index, 1);

  writeTodos(todoList);

  // Print the message
  console.log('Todo deleted');
}

Implementing with classes

// File: src/index.ts
import process from 'process';
import TodoService from './TodoService';

// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;

const todoService = new TodoService();

switch (subcommand) {
  case 'add': {
    const [title, description] = args;
    todoService.addTodo(title, description);
  }
  case 'list': {
    todoService.listTodos();
    break;
  }
  case 'done': {
    const [id] = args;
    todoService.markTodoAsDone(id);
    break;
  }
  case 'undone': {
    const [id] = args;
    todoService.markTodoAsUndone(id);
    break;
  }
  case 'delete': {
    const [id] = args;
    todoService.deleteTodo(id);
    break;
  }
  default:
    // Print help messages
    console.log(`Unknown subcommand`);
    console.log(`Usage: tdr <subcommand> [args]`);
    console.log(`Subcommands: add, list, done, undone, delete`);

    // Exit with an error code
    process.exit(1);
}

// File: src/TodoService.ts
import fs from 'fs';

// Define what a todo looks like
export interface Todo {
  id: string;
  title: string;
  description: string;
  completed: boolean;
}

// Define the TodoService class
class TodoService {
  constructor() {}

  private readTodos(): Todo[] {
    if (!fs.existsSync('todos.json')) {
      fs.writeFileSync('todos.json', '[]');
    }

    const fileConent = fs.readFileSync('todos.json', 'utf-8');
    const todoList = JSON.parse(fileConent) as Todo[];

    return todoList;
  }

  private writeTodos(todoList: Todo[]): void {
    const updatedFileContent = JSON.stringify(todoList, null, 2);
    fs.writeFileSync('todos.json', updatedFileContent);
  }

  public addTodo(title: string, description: string): void {
    const todoList = this.readTodos();

    // Generate a new todo
    const id = Math.random().toString(36).substr(2, 5);
    const newTodo: Todo = {
      id: id,
      title,
      description,
      completed: false,
    };

    // Add the new todo to the list
    todoList.push(newTodo);

    this.writeTodos(todoList);

    console.log('New todo added');
  }

  public listTodos() {
    const todoList = this.readTodos();

    // Print the list of todos
    for (const todo of todoList) {
      const status = todo.completed ? 'X' : ' ';
      console.log(`- [${status}] (id: ${todo.id}) ${todo.title}`);
      if (todo.description) console.log(`\t${todo.description}`);
    }
  }

  public markTodoAsDone(id: string) {
    const todoList = this.readTodos();

    // Find the todo with the given id
    const todo = todoList.find((todo) => todo.id === id);
    if (!todo) {
      console.log('Todo not found');
      process.exit(1);
    }

    // Mark the todo as completed
    todo.completed = true;

    this.writeTodos(todoList);

    // Print the message
    console.log('Todo marked as done');
  }

  public markTodoAsUndone(id: string) {
    const todoList = this.readTodos();

    // Find the todo with the given id
    const todo = todoList.find((todo) => todo.id === id);
    if (!todo) {
      console.log('Todo not found');
      process.exit(1);
    }

    // Mark the todo as not completed
    todo.completed = false;

    this.writeTodos(todoList);

    // Print the message
    console.log('Todo marked as undone');
  }

  public deleteTodo(id: string) {
    const todoList = this.readTodos();

    // Delete the todo with the given id
    const index = todoList.findIndex((todo) => todo.id === id);
    if (index === -1) {
      console.log('Todo not found');
      process.exit(1);
    }
    todoList.splice(index, 1);

    this.writeTodos(todoList);

    // Print the message
    console.log('Todo deleted');
  }
}

export default TodoService;

Implementing Promise based file operations

// File: src/index.ts
import process from 'process';
import TodoService from './TodoService';

// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;

const todoService = new TodoService();

async function main() {
  await todoService.init();

  switch (subcommand) {
    case 'add': {
      const [title, description] = args;
      await todoService.addTodo(title, description);
    }
    case 'list': {
      await todoService.listTodos();
      break;
    }
    case 'done': {
      const [id] = args;
      await todoService.markTodoAsDone(id);
      break;
    }
    case 'undone': {
      const [id] = args;
      await todoService.markTodoAsUndone(id);
      break;
    }
    case 'delete': {
      const [id] = args;
      await todoService.deleteTodo(id);
      break;
    }
    default:
      // Print help messages
      console.log(`Unknown subcommand`);
      console.log(`Usage: tdr <subcommand> [args]`);
      console.log(`Subcommands: add, list, done, undone, delete`);

      // Exit with an error code
      process.exit(1);
  }
}

main();

// File: src/TodoService.ts
import fs from 'fs';

// Define what a todo looks like
export interface Todo {
  id: string;
  title: string;
  description: string;
  completed: boolean;
}

// Define the TodoService class
class TodoService {
  fileName = 'todos.json';

  constructor() {}

  private async fileExists(): Promise<boolean> {
    try {
      await fs.promises.stat(this.fileName);
      return true;
    } catch {
      return false;
    }
  }

  async init(): Promise<void> {
    if (!(await this.fileExists())) fs.promises.writeFile(this.fileName, '[]');
  }

  private async readTodos(): Promise<Todo[]> {
    const fileConent = await fs.promises.readFile(this.fileName, 'utf-8');
    const todoList = JSON.parse(fileConent) as Todo[];

    return todoList;
  }

  private async writeTodos(todoList: Todo[]): Promise<void> {
    const updatedFileContent = JSON.stringify(todoList, null, 2);
    await fs.promises.writeFile(this.fileName, updatedFileContent);
  }

  public async addTodo(title: string, description: string): Promise<void> {
    const todoList = await this.readTodos();

    // Generate a new todo
    const id = Math.random().toString(36).substr(2, 5);
    const newTodo: Todo = {
      id: id,
      title,
      description,
      completed: false,
    };

    // Add the new todo to the list
    todoList.push(newTodo);

    this.writeTodos(todoList);

    console.log('New todo added');
  }

  public async listTodos(): Promise<void> {
    const todoList = await this.readTodos();

    // Print the list of todos
    for (const todo of todoList) {
      const status = todo.completed ? 'X' : ' ';
      console.log(`- [${status}] (id: ${todo.id}) ${todo.title}`);
      if (todo.description) console.log(`\t${todo.description}`);
    }
  }

  public async markTodoAsDone(id: string): Promise<void> {
    const todoList = await this.readTodos();

    // Find the todo with the given id
    const todo = todoList.find((todo) => todo.id === id);
    if (!todo) {
      console.log('Todo not found');
      process.exit(1);
    }

    // Mark the todo as completed
    todo.completed = true;

    this.writeTodos(todoList);

    // Print the message
    console.log('Todo marked as done');
  }

  public async markTodoAsUndone(id: string): Promise<void> {
    const todoList = await this.readTodos();

    // Find the todo with the given id
    const todo = todoList.find((todo) => todo.id === id);
    if (!todo) {
      console.log('Todo not found');
      process.exit(1);
    }

    // Mark the todo as not completed
    todo.completed = false;

    this.writeTodos(todoList);

    // Print the message
    console.log('Todo marked as undone');
  }

  public async deleteTodo(id: string): Promise<void> {
    const todoList = await this.readTodos();

    // Delete the todo with the given id
    const index = todoList.findIndex((todo) => todo.id === id);
    if (index === -1) {
      console.log('Todo not found');
      process.exit(1);
    }
    todoList.splice(index, 1);

    this.writeTodos(todoList);

    // Print the message
    console.log('Todo deleted');
  }
}

export default TodoService;

Summary

Exercises

Database

In the previous chapter, we have implemented a simple command line TODO app. However, the data is stored in a single unstructured JSON file. This means that in order to make some changes on the system we must read the whole file and write the whole file back. This has some drawbacks such as:

• Reading the whole file will be slow if the file is large.
• If file is too large fitting it into memory will be a problem.
• Writing the whole file back will be slow if the file is large.
• Writing to a file is an error-prone operation. (If the system crashes while writing, the file might be corrupted)
• It is hard to query the data.

In order to overcome these problems, we may have two options:

1. Implementing more sophisticated file operations.
2. Using a database.

The first solution will be hard to implement. For this reason, we will use a database in this chapter. However, in preceding chapters, we will also implement a simple file-based database in order to grasp the basics of databases.

Implementing with SQLite

We will use SQLite in this chapter becuase of its simplicity.

-- Create a TODO table
CREATE TABLE todos (title TEXT, description TEXT, done BOOLEAN);

-- We can drop a table by using DROP TABLE statement
DROP TABLE todos;

-- We can update the definition of a table by using ALTER TABLE statement
ALTER TABLE todos ADD COLUMN id INTEGER PRIMARY KEY AUTOINCREMENT;

-- We can insert data into a table by using INSERT INTO statement
INSERT INTO todos (title, description, done) VALUES ('A sample task', 'This is a sample task', 0);

-- We can select data from a table by using SELECT statement
SELECT * FROM todos;
SELECT id, title FROM todos;
SELECT id, title FROM todos WHERE done = 0;
SELECT id, title FROM todos WHERE done = 0 ORDER BY id DESC;
SELECT id, title FROM todos WHERE done = 0 ORDER BY id DESC LIMIT 1;

-- We can update data in a table by using UPDATE statement
UPDATE todos SET done = 1 WHERE id = 1;

-- We can delete data from a table by using DELETE statement
DELETE FROM todos WHERE id = 1;

-- We can delete all data from a table by using DELETE statement without WHERE clause
DELETE FROM todos;

// File: src/index.ts
import process from 'process';
import TodoService from './TodoService';
import TodoSQLiteService from './TodoSQLiteService';

// Get environment variables
const DB_TYPE = process.env.DB_TYPE || 'sqlite';

// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;

async function main() {
  const todoService = DB_TYPE === 'sqlite' ? new TodoSQLiteService() : new TodoService();

  await todoService.init();

  switch (subcommand) {
    case 'add': {
      const [title, description] = args;
      await todoService.addTodo(title, description);
    }
    case 'list': {
      await todoService.listTodos();
      break;
    }
    case 'done': {
      const [id] = args;
      await todoService.markTodoAsDone(id);
      break;
    }
    case 'undone': {
      const [id] = args;
      await todoService.markTodoAsUndone(id);
      break;
    }
    case 'delete': {
      const [id] = args;
      await todoService.deleteTodo(id);
      break;
    }
    default:
      // Print help messages
      console.log(`Unknown subcommand`);
      console.log(`Usage: tdr <subcommand> [args]`);
      console.log(`Subcommands: add, list, done, undone, delete`);

      // Exit with an error code
      process.exit(1);
  }
}

main();

// File: src/TodoSQLiteService.ts
import * as sqlite from "sqlite";
import sqlite3 from "sqlite3";

// Define what a todo looks like
export interface Todo {
  id: string;
  title: string;
  description: string;
  completed: boolean;
}

// Define the TodoService class
class TodoSQLiteService {
  db: sqlite.Database | null = null;

  constructor() {}

  public async init() {
    // Open the database
    this.db = await sqlite.open({
      filename: "todos.db",
      driver: sqlite3.Database,
    });

    // Create the table if it doesn't exist
    await this.db.run(
      "CREATE TABLE IF NOT EXISTS todos (id INTEGER PRIMARY KEY, title TEXT, description TEXT, done BOOLEAN);",
    );
  }

  public async addTodo(title: string, description: string): Promise<void> {
    if (!this.db) throw new Error("Database not initialized");

    await this.db.run(
      "INSERT INTO todos (title, description, done) VALUES (?, ?, ?);",
      [title, description, false],
    );

    console.log(`Added todo: ${title}`);
  }

  public async listTodos() {
    if (!this.db) throw new Error("Database not initialized");

    const todos = await this.db?.all("SELECT * FROM todos;");
    console.log(todos);
  }

  public async markTodoAsDone(id: string): Promise<void> {
    if (!this.db) throw new Error("Database not initialized");

    await this.db.run("UPDATE todos SET done = ? WHERE id = ?;", [true, id]);

    console.log(`Marked todo as done: ${id}`);
  }

  public async markTodoAsUndone(id: string): Promise<void> {
    if (!this.db) throw new Error("Database not initialized");

    await this.db.run("UPDATE todos SET done = ? WHERE id = ?;", [false, id]);

    console.log(`Marked todo as undone: ${id}`);
  }

  public async deleteTodo(id: string): Promise<void> {
    if (!this.db) throw new Error("Database not initialized");

    await this.db.run("DELETE FROM todos WHERE id = ?;", [id]);

    console.log(`Deleted todo: ${id}`);
  }
}

export default TodoSQLiteService;

Implementing Validation

We will add these 3 validation rules to our application:

1. Title must be at least 3 characters.
2. Description must be at least 3 characters.

In order to implement these validation rules we need to add the logic to TodoService.ts file which is our custom database.

Implementing Repository Pattern

Summary

Exercises

Server Side Rendering

Right now, you have a working CLI based TODO app. However you noticed that sometimes you need to access these tasks from your mobile phone or your tablet. In order to do that you need to be able to access your computer from different devices. This means that you need a some kind of server that will be always be running and accessible from the internet. In this challange, you will implement a server side rendered TODO app. This means that you will have a server that will be running on a remote machine and you will be able to access it from your mobile phone, tablet or any other device that has a browser. You will be able to add, delete, update and list your tasks from your browser.

HTTP Server

In order to serve your web page, you need a HTTP server. A HTTP server is a program that listens for HTTP requests and sends HTTP responses. There are a lot of HTTP servers. We will use express.js in this challange. Express.js is a framework that is built on top of node.js. A sample express.js server:

const express = require('express');

const app = express();

app.get('/', (req, res) => {
  res.send('<h1>Hello World</h1>');
});

app.listen(3000, () => {
  console.log('Server is listening on port 3000');
});

Html Forms

An HTML Form (form) is a simple way to collect input from user. Most commonly input is sent to a server to be processed. form element supports common input UI component types such as text, checkbox, radio. Then the values written by the user can be submitted to a server with the submit element.

Template Engine

In order to create dynamic web pages, you need to use a template engine. A template engine is a program that is used to create HTML documents. It takes a template and some data and creates an HTML document according to the template. There are a lot of template engines. We will use handlebars in this challange. A sample handlebars template:

<h1>{%title%}</h1>
<ul>
  {%#each tasks%}
  <li>{%this%}</li>
  {%/each%}
</ul>

XSS Attack

Cross-Site Scripting (XSS) is a security vulnerability in web applications where untrusted data is improperly included in web page content, enabling malicious code execution in the context of other users' web browsers. This vulnerability occurs when web applications fail to adequately validate or sanitize input data, allowing an attacker to inject and execute code that can steal information, manipulate user sessions, or perform unauthorized actions on the affected website.

Implementing Controller-Service-Repository Pattern

Implementing with a template engine

Security

If you are developing a web application,you should be very careful about security. This challenge aims to give some fundamental knowledge about security in web applications. Here are some topics that is expected to learn in this challenge:

• OAuth2.0
• OpenID Connect
• JWT
• Authorization Policy
• Rate Limiting
• JSON Schema
• Validation

Authentication

Authentication is the process of verifying the identity of a user, system, or entity attempting to access a computer system, network, or application. In the web applications this process typically involves the use of usernames and passwords, multi-factor authentication, or other security measures to ensure that only authorized individuals gain access to web-based resources, protecting user data and maintaining system security. Not only for securing the sensitive data, but also for matching a data with a user, a user can only access their own data.

TLS (Transport Layer Security)

CSRF Attack

Cross-Site Request Forgery (CSRF) is a security vulnerability in computer engineering where an attacker tricks a user's web browser into making unauthorized and unintended actions on a different website where the user is authenticated. This occurs when the victim, who is already logged into a web application, visits a malicious website that initiates requests to the target site on the victim's behalf, potentially leading to actions such as changing account settings or making financial transactions without the user's consent. CSRF exploits the trust between the user and the web application, highlighting the importance of proper request validation and the use of security tokens to prevent such attacks.

Encryption

Symmetric encyption Asymmetric encryption

Hashing

Validation

Validation is the process of checking the provided data is in the expected or correct format. For example, if you have a form that takes an email address, you should check if the provided email address is in the correct format. Or you may want to check whether the provided age is a positive number and less than 100. Also, a given json field may contain an in correct type. For example, a field that should be a number may be a string.

In order to check these kind of validations, there are multiple approaches. For instance, you can use assert function provided by Node.js which throws an error if a given condition is not met. Or you can use a library for validating the data. In its simplest form you can use if statements to check the data.

if ((!'name') in req.body) throw new Error('Name is required');
if (typeof req.body.name !== 'string') throw new Error('Name should be a string');
if (req.body.name.length < 3) throw new Error('Name should be at least 3 characters');

Or, you can implement validation by using assert function provided by Node.js which looks a bit cleaner.

import assert from "assert";

...

assert("name" in req.body, "Name is required");
assert(typeof req.body.name === "string", "Name should be a string");
assert(req.body.name.length >= 3, "Name should be at least 3 characters");

JSON Schema

Authorization

Authorization is the process of verifying that a user has access to a resource. This means that, it should not be possible to access a resource which belongs to another user. For example, if you have a task, it should not be possible to delete it if you are not the owner of the task. In order to do that, you need to implement a simple authorization system.

JWT

OAuth2.0

API Rate Limiting

Everything in this world is built with finite resources. if access to the API is unlimited, anyone (or anything) can use the API as much as they want at any time, potentially preventing other legitimate users from accessing the API. Here comes the API rate limiting. API rate limiting is a technique that limits the number of requests that can be made to an API within a specified time period. Here are the general steps conceptually:

1. Identification: Identify the requester. This can typically be done using the IP address, API key, or token.

2. Count Requests: Count the requests made by the identified user or system in a specific time window.

3. Set Limits: Define a limit for the number of requests that can be made in that time window.

4. Block/Allow: Allow: If the number of requests is below the limit, allow the request. Block: If the number of requests exceeds the limit, block the request and send an error message like 429 Too Many Requests.

There are helper libraries that handles all above steps for you. ('express-rate-limit' for Express.js) You can use them in the future.

Testing

Functional Testing with Cypress

Unit Testing with Jest

Integration Testing with

Mocking

Stubbing

Spying

Code Coverage

Realtime Communication

WebSockets

Server Sent Events

gRPC

GraphQL

RPC

REST

JSON-RPC

SOAP

XML-RPC

Webhooks

Idempotency

Caching

Client Side Rendering

Right now, you have a working server side rendered TODO app. Afterawhile, you have recognized that you need a native mobile app for your TODO app. Lets assume that this mobile app is written by another developer. However, he/she needs to access your tasks. In order to do that, you need to create a REST API. This means that you need to communicate via JSON objects. In this challange, you will implement a REST API for your TODO app. Also, you will remove SSR support from your TODO app. This means that you will create a client side rendered TODO app.

In addition to these, also you noticed that you need to attach some files or photos to your tasks. In order to do that, you will use AWS S3. This means that you will upload files to S3 and store their URLs in your database.

In this challenge, it is expected to learn basics of:

• AWS S3 (with presigned URLs)
• File Upload from browser

This challange does not introduce any new feature other than uploading files to S3.

Requirements

You can use libraries like axios and lodash. Also in the backend side, there are no restrictions. You can use any library you want. However, you should not use any React like framework or library. The purpose of this challenge is to learn the basics of client side rendering by using vanilla Javascript.

File Upload and S3

Sending files over HTTP is a common need in almost every web application. In this challange, you will try to upload files to S3. S3 is a cloud storage service provided by AWS. You can upload files to S3 and download them later. We will not use FTP because it is not a good practice to store files in your server. If you would use FTP, horizontal scaling would be a problem. Also, you would need to store files in your server. This means that you would need to pay for storage. However, S3 is a cloud storage service. This means that you will only pay for the storage that you use. Also, you can scale your application easily.

There are two approaches to upload files to S3. The first one is to upload files directly from your backend. This means that your backend will receive a file from browser and upload it to S3. The second approach is to upload files directly from browser. This means that your browser will upload a file to S3. The second approach is better because you don't need to send files to your backend. This means that you don't need to pay for bandwidth.

S3 Presigned Url

If you have an application where users need to upload files to your S3 bucket, you can provide them with a presigned URL to upload a file directly, without them having AWS access. This way you do not need to share AWS credentials or make the file public. The presigned URL is generated by server that has AWS credentials and permission to access the object. The URL embeds the permissions as query string parameters and a signature to ensure they can't be tampered with. The URL is valid for a limited time, defined when the URL is created. After this time, the URL will no longer provide access to the object.

Distributed Systems

ACID

CAP Theorem

SQL vs NoSQL

Database Architecture

Replication

Sharding

DynamoDB

AWS APIGateway and Reverse Proxy

Also, in order to grasp database indexing, you will write a script which will call your API endpoints. This script will be used to benchmark your API. This means that you need to implement a simple benchmarking script. This script will do the following things (please print results to console):

• Insert 10 million tasks and measure the time it takes to insert them.
• List all tasks and measure the time it takes to list them.
• List all done tasks and measure the time it takes to list them.
• List all undone tasks and measure the time it takes to list them.
• Delete all tasks and measure the time it takes to delete them.

Microservices

It is expected to learn these concepts in this challenge:

• Microservice Architecture
• Service Discovery and Registration
• Load Balancing
• Circuit Breaker
• API Gateway
• Event Driven Architecture
• CQRS
• Saga Pattern
• Sync vs Async Communication
• Service Mesh

Virtual Machines

Virtual machines are isolated computing environments that run on a physical machine. The purpose of a virtual machine is running multiple operating systems on a single physical machine. This isolation allows you to run multiple applications on a single physical machine without interfering with each other. EC2 is an example of a virtual machine service.

An hypervisor is a software that runs on a physical machine and manages virtual machines. There are two types of hypervisors: Type 1 and Type 2. Type 1 hypervisors run directly on the hardware and Type 2 hypervisors run on an operating system. KVM is an example of a Type 1 hypervisor and VirtualBox is an example of a Type 2 hypervisor. A host operating system or host machine is an operating system that runs on a physical machine. A guest operating system is an operating system that runs on a virtual machine. A host machine is a physical machine that runs a hypervisor.

Emulation and virtualization are two different concepts used by hypervisors. Emulation is the process of running a software that mimics the behavior of a hardware. This software completely simulates the hardware. As you can imagine, emulation is slow because it has to simulate the hardware. On the other hand, virtualization is the process of running a software / operating system on a virtual machine by running the software directly on the physical hardware. In order to utilize virtualization, the CPU must support virtualization because the isolation is done by the CPU itself. In a virtualized environment, the software again works on a complete isolation without the need of simulating the hardware. If the host machine's CPU architecture is different than the guest machine's CPU architecture, then the hypervisor should use emulation. On the other hand, if virtualization is possible, virtualization should be preferred.

AWS IAM

AWS Roles

AWS SSO

AWS CLI

AWS EC2

EC2 is the short for Elastic Compute Cloud. It is an Aws microservice that provides virtual machines. A cloud hosted virtual machine is useful because you can access it from anywhere/anytime. A connection is needed to access the virtual machine. Opening a connection session is called SSH for Linux systems.

AWS RDS

In order to host a DBMS, you need to host the actual database software in a system. This is where AWS RDS comes into play. AWS RDS is a service that provides DBMS instances. As you expose your DBMS to the internet, you can access it from different sources.

Containers

A computer program is composed of two main parts: code and data. Code is the instructions that are executed by the computer. Data is the information that is processed by the code. In order to run a program, the code must be loaded into the memory. The memory is a temporary storage that is used by the CPU to store the code and data.

Code part of a program is contained in a file which is called an executable file (.exe in Windows, no common extension in Linux and MacOS). An executable file may contain cpu instructions and data that will be loaded into the memory. If the program does not need to load some data into memory before running, then the executable file can run some instructions in order to read the data from a file. Or if the program needs to store a data in a persistent way, then the program can write the data to a file. These operations are done by using operating system's system calls.

Therefore, a large program can be considered as a collection of executable files and data files (sometimes called assets, resources, etc.). However, an executable file may need a library in order to run. A library is a collection of functions that can be used by a program. These files are not executable files, but they are needed by the executable file in order to run (in Windows, they have .dll extension, in Linux they have .so extension, in MacOS they have .dylib extension). In order to run a program, the operating system must load the executable file and the libraries into the memory or the library can be loaded into the memory when the program needs it.

A static binary is an executable file that contains all the code and does not need any library in order to run. On the other hand, a dynamic binary is an executable file that needs a library in order to run. If the library is not found in the system, then the program will not run and will give an error. Also, if a library is updated, then all the programs that use this library also will be affected this change. This may cause a problem because the program may not work as expected.

A container is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings. Therefore every library that is needed by the program is included in the container. This means that the program may run in any environment that shares the same operating system kernel. This is because the container does not contain the operating system itself, but it contains the all other parts that are needed by the program.

This approach solves the problems that are mentioned above. However, it is not enough to bundle all dependencies of a program into a single package. Also we need an isolation mechanism in order to run multiple containers on the same machine in a secure way. This isolation is done by the operating system kernel. The operating system kernel is the core of the operating system that manages the hardware and provides services to the programs. The kernel is responsible for managing the memory, the CPU, the file system, the network, etc. Indeed, each process (a running instance of a program) is isolated from each other in memory. This means that a process cannot access the memory of another process by default. However, filesystem, network, etc. are shared between processes. In order to isolate these parts. Linux introduced namespaces and cgroups. By utilizing these features of the Linux kernel, containers can work on a complete isolation.

Therefore, contaieners may be used for isolation without the need of virtual machines. This comes with a significant performance improvement. For this reason, containers are widely used in the industry. However, in order to work with containers, you need some tools to build container images, run containers, and manage containers. These tools generally comes in a bundled package called container runtime.

Docker

Docker is one of the most popular container runtimes in the industry.

WIP: Container Image WIP: Container Registry WIP: Dockerfile

Docker Compose

App Runner

Serverless

AWS Lambda

CI/CD

DevOps

DevOps is a set of practices that combines software development (Dev) and IT operations (Ops). It aims to shorten the systems development life cycle and provide continuous delivery with high software quality.

Github Actions

Github Secrets and Github Variables

Github Codeowners

PR Environments

Infrastructure as Code

Your app works perferctly. However, when you want to make a simple change, you need to deploy your app manually. If you are a lazy developer, then this challange is for you. In this challange, you will implement a continuous deployment pipeline for your TODO app. This means that, when you push your code to your repository, your app will be automatically deployed to your server. Also, you may notice that, when you made a change you do not have an environment to test your changes. In this challange, you will also separate your environments.

In this challenge, it is expected to learn basics of:

• Continuous Deployment with Github Actions
• Infrastructure as Code with Terraform
• Environment
• AWS SSO
• AWS CLI

Continuous Deployment is a software development practice where code changes are automatically built, tested, and to an environment. This means that, when you push your code to your repository, your app will be automatically deployed to your server.

Environment

An environment is a place where your app is running. Yo

Infrastructure as Code

Terraform

Container Orchestration

Right now, you have a working server side and client side rendered TODO app. Lets imagine that your app has become very popular and you have thousands of users. How you can handle this load? In this challenge, you will learn how to scale your application's computation.

Isolated Environments

A common need in software development is to run your app in an isolated environment. This is especially important in production environments. There are some reasons why you need to run your app in an isolated environment:

1. Security: Another app that is running on the same machine can affect your app by using file system, network, or memory. This should be avoided because your app may behave unexpectedly.
2. Performance: Also another app that is running on the same machine can affect your app's performance. For instance, another app can consume all the memory or CPU, which can slow down your app.
3. Versioning: You may need to run multiple versions of an app on the same machine.

Also, app is not just your backend code. It is also may be a database, a message queue, a cache, a search engine, etc. In order to solve these problems, there are two common approaches: virtual machines and containers.

ECS and ECR

Kubernetes