Preface
According to CHAOS Report, more than 50% of the software projects are failing. These projects are either canceled or delivered late or over budget. The reasons of failing projects may be technical or non-technical. One of the main technical reasons of failing projects is the low quality code. Because of the low quality code, maintaining the project and adding new features to the project is very hard. Also, the project may have a lot of bugs. These bugs may cause security vulnerabilities or data loss. So, writing a high quality code is very important for the success of a software project.
The Problem
High Quality Software
Writing a high quality code is not an easy task for developers and it requires a lot of effort and time. If a project has a long life cycle and its requirements are changing frequently, high quality code is a must not an option. For this reason, developers must strengthen their coding skills and learn new technologies in order to write high quality code.
Also, implementing a software project with a team is completely different than implementing a software project alone. In a team, developers need to follow some rules and conventions in order to work together effectively. Also, they need to use some tools and technologies to communicate with each other and to manage the project. For this reason, developers need to learn some concepts, technologies and tools in order to work in a team.
This book focuses on these challenges and its main goal is writing a good quality code in a team or a personal project. I personally do not believe that if you do not understand the underlying concepts and technologies, you can not write a good quality code. Yes, you can write a code that works but it will probably not be a high quality code. So, in this book, I will try to explain the underlying concepts and technologies in detail.
Structure of the Book
First of all, in order to learn these concepts and technologies, we will invent a problem and then we will try to solve it. Because, at its core software development is about solving problems. It is not about the problem itself. The problem always come from the real world. As developers, we do not need to explore the new problems. However we must understand the problem in order to solve it. Without understanding the problem, trying to solve it is just a waste of time.
After defining the problem, we will refresh our knowledge about the underlying concepts and technologies. Then, we will try to solve these problems in a right way. We will try to write a clean code that is easy to read and understand by following the best practices and conventions.
Some recommendations
The book covers a lot of topics and reading the book like a novel is not a good idea. Instead of that, you may read the some parts of the book again and again. For this reason, you can skip the chapters that you already know, but my advice is to read them because you may learn something new or notice something that you did not notice before.
It is very important to practice what you learn. So, while reading the book you should try to implement the examples in the book by yourself. I personally believe that, just reading the book will not be enough to grasp the whole book. You need to practice what you learn in order to understand the concepts deeply.
Also you can use the book as a reference. You can search for a topic in the book and then read the related chapter. You can
If you are in a trouble, please make sure that you have understood the concepts completely. If you are not sure, you can read the related chapter again. If you are still stuck, you can just google it. Also, you can use AI tools like ChatGPT to get help. However, please do not just copy and paste the code. Try to understand the code and then try to solve the challenge by yourself. Also, you can support the book with some online courses, YouTube videos, articles, books and any other resources. In each chapter you can find additional resources that you can use to learn the topic in detail.
Also, while writing your code, ask your self these questions:
- Is my code clean and readable?
- May i simplify the code or the solution?
- Is my code following the best practices and conventions?
- Is my code well-documented? (Comments, README etc.)
- Is my code tested? (Unit tests, integration tests etc.)
- Is my code secure?
- Is my code scalable?
- Is my code maintainable?
- Is my code consistent? (Naming conventions, code style etc.) (You can use linters and formatters to check this)
Assumptions and Requirements
First of all, in order to organize the content of the book, I need to know the audience of the book. I tried to write the book for the developers who are familiar with the programming concepts and technologies. However, I tried to explain the concepts in detail. So, if you are a beginner, you can still read the book. However, you may need to read some parts of the book again and again. For this reason, I have these assumptions about the audience of the book:
- You know a programming language. (You do not need to be an expert but you should know variables, conditional, loops, functions etc.)
- You know what an object oriented programming is. (You do not need to be an expert but you should know what a class, object, inheritance etc. is.)
Concepts and Technologies
Here are the concepts and technologies that we will be covered in the book:
- Command Line Interface Basics
- Server Side Rendering and Templating Engines
- Basic DOM Manipulation
- Client Side Rendering with React
- HTML & CSS
- Markdown Documentation
- Typescript and JavaScript
- Continuous Deployment (with GitHub Actions)
- Continuous Integration (with GitHub Actions)
- Cloud Computing (with AWS ECS, Fargate and Lambda)
- Cloud Databases (with AWS RDS, DynamoDB)
- Cloud Storage (with AWS S3)
- CDN (with AWS CloudFront)
- Authorization
- Authentication
- OAuth2, JWT and OpenID Connect
- Microservices Architecture
- RESTful API Design
- Clean Architecture / Hexagonal Architecture / Onion Architecture / Ports and Adapters
- Clean Code
Computer Basics
A computer is an automated machine that can perform tasks by following set of instructions. A computer may be a desktop, tablet, smartphone or a toaster. These computers may be an electronic device or a mechanical device. The idea behind the computers is not bound to the electronic devices.
A computer may work as an analog computer or a digital computer. Analog computers are the computers that work on the continuous data on the other hand digital computers work on the discrete data. Digital computers are the most common computers that we use in our daily life because they have an huge advantage over the analog computers when it comes to the accuracy and the precision.
Lets give an example; lets say that we have a computer that just adds two numbers. If we would use an analog fluid computer, the numbers would be represented by the pressure of the fluid. Therefore, the adding machine would create a pressure in its output that will be match the sum of the input pressures.
On the other hand, if we would use a digital computer, the numbers would be represented by the binary numbers. In order to represent binary numbers as a pressure, we need to pick constant pressure values for 0 and 1.
Parts of a Computer
Binary Numbers (0s and 1s)
In the digital world, everything is represented by the binary numbers. Binary numbers are the numbers that are represented by the 0s and 1s. The binary numbers are the most basic form of the data representation in the digital world. The binary numbers are used to represent the numbers, characters, images, sounds and videos.
You may think that how is it possible to represent everything with just 0s and 1s. This question is a good question and the answer is not trivial. Lets see;
Lets we want to store whether a light is on or off then we can represent the light is on with 1 and the light is off with 0. However, lets say that we want to store the color of the light. Lets assume that the light have 3 different colors then we can represent the state of the lisht bulb with 2 bits. Therefore, there are 4 different possible states;
Color | Binary |
---|---|
Off | 00 |
Red | 01 |
Green | 10 |
Blue | 11 |
As you can see we can concatenate the bits to represent more complex data. These groups of bits have a name;
Number of Bits | Name |
---|---|
1 | Bit |
4 | Nibble |
8 | Byte |
16 | Word |
32 | Double Word |
64 | Quad Word |
However, generally people use hexadecimal numbers to represent the binary numbers because they have a more compact form. Hexadecimal numbers are the numbers that are represented by the 0-9 and A-F. Lets see the hexadecimal representation of the binary numbers;
Binary | Hexadecimal |
---|---|
0000 | 0 |
0001 | 1 |
0010 | 2 |
0011 | 3 |
0100 | 4 |
0101 | 5 |
0110 | 6 |
0111 | 7 |
1000 | 8 |
1001 | 9 |
1010 | A |
1011 | B |
1100 | C |
1101 | D |
1110 | E |
1111 | F |
To make the distinction between the hexadecimal and the decimal numbers, the hexadecimal numbers are generally prefixed
with 0x
. For example, the hexadecimal number 0x2A
represents 8 bits of data in hexadecimal form. Also for the binaries
the prefix 0b
is used. For example, the binary number 0b110
represents 3 bits of data in binary form. If we are using
decimals then we do not need to use any prefix.
Lets say that we want to store a number by using the binary numbers. Simply we can convert each number to its binary representation. For example, the number 5 is represented by 101 in binary.
However, this method to represent the numbers is not enough to represent the negative numbers. Therefore, we need to invent a new method to represent the negative numbers. The simplest method to represent the negative numbers is making the most significant bit as the sign bit (leftmost bit). If the sign bit is 1 then the number is negative, otherwise the number is positive. Lets look at some examples;
Logic Gates and Transistors
CPU
Memory
Instruction Set Architecture (ISA)
I/O Devices
Summary
Exercises
Computer Networks
OSI Layers
MAC
IP
UDP
TCP
DNS
Secure Communication
Summary
Exercises
Extra Resources
Operating Systems
At the early days of computers, there was no operating system. People were using computers by writing machine code directly. However, with this method computers were executing only one program at a time. In order to run another program, people were needed to stop the current program and load the new program into the memory. This was a very inefficient method. Although, this method is not suitable for today's computers it is still used in some embedded systems. For instance, your microwave oven has a microcontroller that runs only one program at a time. It is not needed to run multiple programs at the same time. Operating systems provide a way to run multiple programs at the same time. This is the main reason why operating systems were invented. These programs sometimes called applications.
Another problem with this method is that people were needed to write programs for each hardware combination. For instance, if you would want to run Excel on a computer, you would need a version of Excel that is written for your hardware specifically. This is a big problem because software companies need to write a program for each hardware combination which may be thousands of different combinations.For embedded systems, this is not a problem because the hardware is fixed. However, for general-purpose computers, this is a big problem. In order to solve this problem, operating systems were invented. Most operating systems provide a layer of abstraction between hardware and software. Therefore, the only part that depends on hardware is the operating system's kernel. All applications that needs to access hardware can use the kernel functions to access hardware. These functions are called system calls. Therefore, applications do not need to know the hardware details.
Also most operating systems provide a way to manage files. They provide a file system that is used to store files. This file system is a way to store files in a structured way.
Kernel
Executable Files and Processes
Machine code is a set of binary numbers that can be understood by a computer's CPU. It is the lowest level of programming language that a computer can understand. A program is indeed a machine code. However, it also contains some metadata like the name of the program, the version of the program, the author of the program, etc. Therefore a program is a file that contains machine code and some metadata.
In Linux, programs use ELF (Executable and Linkable Format) file format. Therefore, when you want to run a program in Linux, the kernel will read the ELF file and load it into right memory locations. After loading the program into memory, the kernel will jump to the entry point of the program. At this point, CPU will execute instructions of the program instead of the kernel. This is how a program is started in Linux.
On the other hand, a program can be a script file. If a program is not a binary ELF file, it is a script file. A script file is a text file that contains some unspecified code. This code can be written in any programming language. However, it should be a language that is installed on your system.
Command Line Interface
There are 2 common approaches to interact with a computer. The first one is GUI (Graphical User Interface) and the second one is CLI (Command Line Interface). GUI is a visual interface that you can use to interact with your computer. Command line interface is a text-based interface that you can use to interact with your computer. In CLI, you are giving a command to your computer and the computer runs it and gives a result to you by printing it to the screen. There are a lot of commands that you can use in CLI. Here are some of them:
A cli command is nothing more than a program that you can run from your terminal or shell. Each cli command is an
executable file located in your PATH
. When you run a command, your shell will search for the executable file in your
PATH
and if it finds it, it will run it. So, you need to put your executable file in your PATH
in order to run it
from anywhere. You can check your PATH
by using echo $PATH
command. It is a list of directories separated by :
.
You can put your executable file in one of these directories.
Let's look at how ls
command works. When you write ls
in your terminal, it will try to find an executable file named
ls
in your PATH
. It will check each directory in your PATH
one by one. If it finds it in the first directory,
it will run it directly. If it does not find it in the first directory, it will check the second directory and so on. If
it does not find it in any of these directories, it will give an error. You can check where ls
command is located
by using which ls
command. It will give you the path of the executable file. Also, terminal will check the file
permissions. If the file is not executable, it will give an error. For this reason you need to know the concept of
users, user groups and file permissions.
There are two types of executable files. The first one is a binary ELF file.
The second one is a script file. A script file is a text file that contains shebang
at the beginning of the file. Shebang is a special comment that starts with #!
. It tells the shell which program to
use to run the script. For instance, if you write #!/bin/bash
at the beginning of your script file, shell will use
bash
to run your script. If you write #!/usr/bin/env node
, terminal will run env
command with node
as the first
argument. And as a second argument, it will give your script file's path. So, env
command will run node
with your
script file's path as the first argument. Please use shebang at the beginning of your script file. Otherwise, your
script will not run.
In the challenge, you will implement an argument parser in order to parse the arguments that are given to your program.
You can get these arguments by using process.argv
array. It is an array that contains the arguments that are given to
your program. You need to follow these subcommands in order to implement your TODO app:
- list:
mytodo list
alsomytodo
(default subcommand) (--done
and--undone
flags should be supported) - add:
mytodo add "A sample task description"
- done:
mytodo done 1
- undone:
mytodo undone 1
- delete:
mytodo delete 1
- update:
mytodo update 1 "A new task description"
Standard IO, Exit Code, Environment Variables and Arguments
Each program ran from the terminal has an exit code. This exit code is just a number between -128 and 127. It is used to
indicate the status of the program. If the program runs successfully, it will return 0. If it fails, it will return a
non-zero number (You can use any non zero number between -128 and 127). You can check the exit code of the last ran
program by using echo $?
command. It will print the exit code of the last ran program. Exit codes are very important
to check the status of a command.
Also, each program has 3 standard streams in order to interact with the user. A stream is a just a file that you can
write to or read from. There are 3 standard streams in a program. These are stdin
, stdout
and stderr
. stdin
is
the standard input stream. It is used to read input from the user (keyboard). stdout
is the standard output stream.
It is used to write output to the user (screen). stderr
is the standard error stream. It is used to write error
messages to the user (screen).
However, also a command can be started with some arguments. These arguments are called command line arguments. Arguments
are just an array of strings. And your arguments should be separated by a space. For instance, if you write ls -l -a
in your terminal, ls
command will be started with 3 arguments. The first argument will be ls
, the second argument
will be -l
and the third argument will be -a
.
Basic Commands
In linux, there is
Each process in linux, has a working directory. A working directory is just a directory that the process is running in.
We can print the current working directory by using pwd
command. We can list all the files in the current directory
by using ls
command. Optionally, we can use;
-l
flag to list files in a long format-a
flag to list all files including hidden files-h
flag to list files in a human readable format-r
flag to list files in reverse order-t
flag to list files by modification time-S
flag to list files by size-R
flag to list files recursively
We can change the current working directory by using cd <path>
command. We can create a new directory by using
mkdir <directory-name>
command and remove a directory by using rmdir <directory-name>
command.
Users and Groups
File System
Permissions
Processes
Package Managers
Shell Scripting
Environment Variables
Also, there is a concept of environment variables. Environment variables are just key-value pairs that are stored in your
shell. These variables are used to get some common configuration values across different programs. For instance, you can
use EDITOR
environment variable to get the editor that you want to use. You can use PATH
environment variable to get
the directories that you want to search for executable files. Do you see the pattern? Your shell uses PATH environment
variable to search for executable files. You can use process.env
object to access these environment variables in your
program. In this challenge, you will use TODO_USERNAME
environment variable to get the assignee of the TODOs.
Stdin, Stdout, Stderr
Exercises
- Install Ubuntu on a virtual machine and explore it.
Version Control
Writing code is a complex and continuous process. For this reason, your code will change over time. In order to keep track of these changes, you will need a tool manage the versions of your code. This is where version control systems come into play. A version control system is a tool that helps you to manage the changes in your code. It allows you to keep track of the changes that you made in your code. Also, it allows you to revert your code to a previous state if something goes wrong. In addition to these, it allows you to collaborate with other developers. You can work on the same codebase with other developers without any conflicts. Also, you can see who did what in the codebase. This is very useful when you are working with a team.
There are many version control systems in the industry. Some of them are centralized and some of them are distributed. Centralized version control systems have a central server that stores the codebase. Developers can clone the codebase from the server and they can push their changes to the server. However, this approach has some drawbacks. For instance, if the server goes down, then no one can access the codebase. Also, if the server is slow, then the developers will have to wait for the server to respond. In addition to these, if the server is hacked, then the codebase can be stolen. These are some of the drawbacks of centralized version control systems.
Distributed version control systems are more popular in the industry. In distributed version control systems, every developer has a copy of the codebase on their local machine. This is very useful because you can work offline and you can commit your changes to your local repository. Also, you can push your changes to the remote repository when you are online. This approach is more secure and faster than centralized version control systems. Also, it is more reliable because you have a copy of the codebase on your local machine.
Git
Git is a version control system that is widely used in the industry. This book especially focuses on Git because of its popularity. Git is a distributed version control system which means that every developer has a copy of the repository on their local machine. This is very useful because you can work offline and you can commit your changes to your local repository. Also, you can push your changes to the remote repository when you are online.
Everything in git starts with a repository. A repository is a place where you store all of the changes and versions of
your project. A repository can be located on your local machine and/or on a remote server. You can create a repository
by running git init
command in your project directory. This command will make the current directory a git repository.
When you run the command, you will notice that it will not do any changes other than creating a .git
folder in your
project directory. Therefore, git stores everything in .git folder. This folder is a hidden folder and you should
not directly modify it if you do not really know what you are doing.
After creating a repository, you can start tracking your files in your project directory. The directory that you are using for your project is called working directory in git. Working directory is completely separate from the repository. Git itself may create, modify or delete files in the working directory according to the commands that you run. Also some commands may affect your repository. Therefore we have two different type of data storage behind the scenes. One is the working directory and the other is the repository.
The synchronization between the working directory and the repository is one of the main concepts in git. When you make
changes in your working directory, you should stage them before committing. Staging is the process of preparing your
changes to be committed. A commit is a snapshot of your changes. It is like a save point in a game. We can add a
file to the staging area by running git add <file>
command. After collecting all the changes that we want to commit,
we can run git commit -m "Commit message"
command to commit the changes. The -m
flag is used to add a message to the
commit. This message should describe what you did in this commit.
It is very important to write meaningful commit messages.
We can see the current status of the repository and the changes that we made by running git status
command. This
command will show us the files that are modified, staged or not staged. Also, we can see the history of the repository
by running git log
command.
# Create a directory
mkdir git-done-right
cd git-done-right
# Initialize a git repository
git init
# Create a file (you may create your files first and then run the init command after that)
echo "Hello, World!" > hello.txt
# Add the file to the staging area (Which means that we want to include all the changes in this file to the next commit)
git add hello.txt
# Lets check the status of the repository
git status
# Commit the changes
git commit -m "Initial commit."
# Check the status again
git status
# Make some changes
echo "\n" >> hello.txt
echo "How are you?" >> hello.txt
# Check the status again
git status
# Add the changes to the staging area
git add hello.txt
# Commit the changes
git commit -m "Added a new line."
# See all the commits
git log
We can imagine our changes in the repository like a chain. Every commit is a link in this chain. We can go back to a
previous commit by running git checkout <commit-hash>
command. The commit-hash
is the hash of the commit that you
want to go back. You can see the commit hashes by running git log
command. After going back to a previous commit, you
can go back to the latest commit by running git checkout master
command. The master
is the name of the branch that
we are currently working on. We will talk about branches later. git checkout
command modifies the working directory
according to the commit that you specified. Therefore, you should be careful when you are using this command. If you
have uncommitted changes in your working directory, then you may lose them.
We have two commits right now and our commit history is looks like this:
gitGraph commit id:"A" commit id:"B"
Lets imagine that we want to do some experiments with our code and we will decide later whether we will keep them or not. In this case, we may utilize branches. A branch is a pointer to a specific commit. At this point, this may be very confusing. For this reason, we should not forget that each commit has a parent commit which refers to the previous commit.
flowchart RL A-- parent -->NULL; B-- parent -->A;
However, instead of drawing the parent pointer explicitly, like the above, we will use git diagrams to represent the branches. In this case, you should keep in mind that each commit is represented by a circle and the parent commit is represented is the commit that is located on the left side of the commit and connected to the commit with an edge.
When we create a branch, git will create a new pointer to the commit that we are currently on. This pointer is called a
branch. We can create a branch by running git branch <branch-name>
command. After creating a branch, we can switch
to the branch by running git checkout <branch-name>
command. This command will switch to the branch that you specified.
gitGraph commit id:"A" commit id:"B" branch experiment-1 commit id:"C" checkout main branch experiment-2 commit id:"D" checkout main merge experiment-2
At this point, our repository is completely stored on our local machine. If we want to share our code with others, we should store them in a place where others can access it. This place is called a remote repository. A remote repository is a repository that is stored on a server which can be accessed by other developers. The server may be publicly accessible or it may be a private server.
In order to work with remote repositories, the first step is to introduce the remote repository to our local repository.
We can do this by running git remote add origin <remote-url>
command. This command will add a remote repository to our
local repository. We may define multiple remote repositories for our local repository and for this reason, we are
providing a name for the remote repository. This name is called an alias. In this case, we are using origin
as an
alias. This is a common convention in the industry. You can use any name that you want. After adding the remote
repository, we can push our changes to the remote repository by running git push origin master
command. This command
will push our changes to the remote repository. The master
is the name of the branch that we are pushing. We will talk
about it later.
# Add remote repository
git remote add origin https://github.com/cebecifaruk/git-done-right
# Push the changes to the remote repository
git push origin master
The push command simply sends all the commits that are not in the remote repository to the remote repository. Therefore, in the remote repository, we have all the knowledge that is required to build the current working directory of the project. This means that we can also get all the commits from a different machine or in a different directory. In order to "download" a project from a remote repository, we should take these steps:
# Create a directory
mkdir git-done-right-clone
# Change the directory
cd git-done-right-clone
# Initialize a git repository
git init
# Add the remote repository
git remote add origin
# Pull the changes from the remote repository
git pull origin master
However, instead of doing all the steps manually, git provides a command to do all these steps in one command. This
command is git clone <remote-url>
. This command will take all the steps that we mentioned above and it will create a
new directory with the name of the repository.
Git Behind the Scenes
Github
Github is a web-based platform that is built on top of git where you can store your repositories and collaborate with other developers. It is a very popular platform in the industry. It provides many features that are not available in git itself.
You can use Github personally by creating a personal account or you can use it as an organization by creating an account for your organization. However, instead of creating an account for an organization, you can create an organization in Github. An organization is just a group of users at the end of the day. You can add users to your organization and you can manage their permissions. Also, you can create teams in your organization. A team is another group of users that are in the organization.
In addition to these, Github is like a social media platform for developers. You can follow other developers, star their repositories and watch their repositories. Starring a repository means that you like the repository. Watching a repository means that you want to get notifications about the repository. You can also fork a repository. Forking a repository means that you are creating a copy of the repository in your account. You can make changes in the forked repository and you can create a pull request to the original repository. A pull request is a request to merge your changes to the original repository. The owner of the original repository can accept or reject your pull request.
In Github, you can track the tasks that you need to do, and these are called issues. Issues may be a bug, a feature request, a question etc. You can assign issues to other developers and yourself. Issues are organized per repository. If you are working on multiple repositories, you can use projects to organize your issues. Projects are like a kanban board where you can organize your issues in columns.
Also, as you can store your documentation in the repository, you can also store your documentation in repository's wiki. A wiki is a place where you can store your documentation in markdown format. You can create pages, link them to each other and you can organize them in a tree structure.
Monorepo vs Polyrepo
Best Practices
Your commits should describe what you did but it should not consists of only changes. For instance README file updated
or x file changed
messages are not good commit messages. Instead, you can use Added a new section about x
for
README file updated
or Added a new function to do y
for x file changed
. Therefore, it should give an idea about
what you did. It is not needed to list all crud operations that you did because they are already listed in the
commit itself.
Also please be consistent between your commit messages. For instance, if you use imperative mood for your commit messages, you should use it for all of your commit messages. If you use past tense, you should use it for all of your commit messages. Also syntacticly they should be consistent. For instance, you can start with a big letter and you can put a dot at the end of the message. But you should be consistent between your commit messages.
It is very important to keep git history clean. You should not commit files that are not related to your project.
For instance, you should not commit your node_modules
folder, IDE files, temporary files etc. You should only commit
files that are related to your project. Also, git stores your history. So, you should not grow your git repository size
by committing unnecessary files. PLease be careful about binary files. Git is not good at storing binary files. So, you
should not commit binary files (If you really need them, then you can). But please be careful about them.
Another thing is using .gitignore
file to ignore files that you don't want to track. For instance, you don't need to
track node_modules
folder. You can ignore it by adding node_modules
to .gitignore
file. However, instead of
writing it manually, you can use gitignore.io to generate .gitignore
file for your project.
Summary
Exercises
Extra Resources
Internet
Hyper Text Markup Language (HTML)
A markup language is a language that is used to annotate text. The reason why it is called markup language is that you are marking up the text. Why we need to annotate text? Lets sey that we want to write a book. We need to add some formatting to the text. For instance, we may need to make some words bold, italic or underline. Also we may need to add some images, tables, lists etc. to the text. However, at the and of the day, we need to store the text in a text file. So, we need a way to annotate the text. This is where markup languages come into play. There are a lot of markup languages. For instance:
- Markdown
- Latex
- HTML
- XML
HTML is different from other markup languages. Because you can view the result in a web browser. It is used to create
web pages. A web page is a text file that contains HTML tags. A tag is a special text that starts with <
and ends with
>
. There are two types of tags. The first one is a self closing tag. It does not have a closing tag. For instance,
<br>
is a self closing tag. The second one is a normal tag. It has a closing tag. For instance, <p>
is a normal tag
and </p>
is its closing tag.
Historically, HTML was used to create static, not styled web pages. If you want to style your web page, you need to use CSS. CSS is a language that is used to style HTML elements. However, in order to create dynamic web pages, you need to use a programming language. This is where javascript comes into play. Javascript is a programming language that is used to create dynamic web pages.
However, also you can serve dynamic web pages with a trick called server side rendering. This means that you can create a web page with a programming language and send it to the client. This is what you will do in this challange.
Cascading Style Sheets (CSS)
Hyper Text Transfer Protocol (HTTP)
In order to serve your webpage, you need a some kind of protocol. This protocol is called HTTP. HTTP is a protocol that is used to transfer hypertext. Hypertext is a text that contains links to other texts. For instance, a web page is a hypertext. It contains links to other web pages.
HTTP is a request-response protocol. This means that a client sends a request to the server and the server sends a
response to the client. A request contains a method, a path and list of headers. A response contains a
status code, some headers and a body. A method is a verb that tells the server what to do. For instance, GET
method
is used to get a resource from the server. A path is a string that tells the server which resource to get. For instance,
/
path is used to get the root resource. A header is a key-value pair that contains some information about the
request. For instance, Content-Type: text/html
header tells the server that the body of the request is a HTML
document. A status code is a number that tells the client what happened. For instance, 200
status code means that
everything is OK. A body is a string that contains the response. For instance, if the response is a HTML document, the
body will contain the HTML document.
Sample HTTP request:
GET / HTTP/1.1
Host: localhost:3000
User-Agent: curl/7.64.1
Accept: */*
As you can see the first line contains the method (GET
), the path (/
) and the HTTP version (HTTP/1.1
) separated by
spaces. The first line is called request line and the other lines are called headers. These lines are separated
by \r\n
. After sending all headers, in order to tell the server that the headers are finished, you need to send an
empty line. After that, you can send the body of the request. In this case, the body is empty.
Sample HTTP response:
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: text/html; charset=utf-8
Content-Length: 25
<h1>Hello World</h1>
As you can see the first line contains the HTTP version (HTTP/1.1
), the status code (200
) and the status message
(OK
) separated by spaces. The first line is called status line and the other lines are called headers. These
lines are separated by \r\n
. After sending all headers, in order to tell the client that the headers are finished,
you need to send an empty line. After that, you can send the body of the response. In this case, the body is
<h1>Hello World</h1>
which is an HTML document.
TLS/SSL
API
REST API
REST (Representational State Transfer) is an architectural style for developing web services. REST is popular due to its simplicity and the fact that it builds upon existing systems and features of the internet's HTTP in order to achieve its objectives, as opposed to creating new standards, frameworks and technologies. REST is a stateless client-server architecture where web services
Javascript
Browsers use JavaScript to make web pages dynamic and interactive. JavaScript allows to manipulate the content, structure, and style of a web page in real-time, respond to user inputs, make asynchronous requests to the server, and much more. We are going to cover manipulating the web-content in this challenge. The JavaScript engine in the browser parses the JavaScript code. (V8 Engine for Chrome, SpiderMonkey for Firefox) The parsed JavaScript code is converted into machine code and executed. The JavaScript code can manipulate the DOM, making real-time changes to the web page content, structure, or style. JavaScript is single-threaded, but it can handle asynchronous operations using the event loop and callback queue. When an asynchronous operation is encountered, it’s offloaded, and its callback is placed in the callback queue. The event loop continuously checks if the main thread (where the synchronous code runs) is free. If it is, the event loop takes the next callback from the callback queue and executes it.
DOM
The DOM is a programming interface for HTML and XML documents. It represents the structure of a document as a tree of objects. Each node in this tree corresponds to a part of the document, for example, an element, an attribute, or text content. Mechanism that browsers use to convert HTML into DOM is below:
- The browser parses the HTML content line by line.
- It converts HTML elements, attributes, and text into objects.
- These objects are structured in a tree-like form, mirroring the nested structure of the HTML tags.
- Each object represents a part of the page and contains properties and methods that you can manipulate using JavaScript.
Modules
Writing client side rendered single page applications is not an easy task and it is not practical to write everything in one file. For this reason, you should split your code into multiple pieces. For this reason, we are using modules. A module is a piece of code that hides or exposes some functionality, and most importantly, it is reusable. Therefore, it is a collection of functions, variables, and classes that are bundled together. These functions, variables, and classes are not accessible from outside the module unless they are explicitly exported. In this way, you can hide the internal implementation details of a module and expose only the necessary parts. This is called encapsulation. Historically, there are several ways to define modules in JavaScript: CommonJS, AMD, UMD, and ES Modules. Lets examine them one by one.
In CommonJs modules, your script work in a context which has a predifined set of variables. You can use require
function to import a module and module.exports
object to export a module. For instance, if you want to use
a library called lodash
, you can import it like this:
const _ = require('lodash');
require
function is a synchronous function that loads the module if it is not loaded before and returns the module
object. If you want to export a function from a module, you can use module.exports
object like this:
module.exports = function () {
console.log('Hello World');
};
Here is an example of a CommonJS module:
// math.js
function add(a, b) {
return a + b;
}
module.exports = {
add,
};
// index.js
const math = require('./math');
console.log(math.add(1, 2));
CommonJS modules are synchronous. When you import a module, it is loaded immediately. This is not a problem in server side applications. However, in client side applications, it is a problem because you have to load all the modules before the application starts. For this reason, CommonJS modules are used by Node.js, not in the browser.
Fetch and XMLHttpRequest API
There are multiple ways to send HTTP requests from browser. The first one is to use fetch
API. This is a native API
and it is supported by all browsers. The second one is to use XMLHttpRequest
API. This is an old API and it is
supported by all browsers. The third one is using a third party library like axios
. This is a library that you can
use to send HTTP requests from browser. These libraries actually wrap XMLHttpRequest
or fetch
API. Third party
libraries generally provide easier to use APIs. However, they are not native APIs. This means that you need to include
them in your project. Also, they increase the size of your project. It is recommended to use axios
API. However,
if you try to use fetch
and XMLHttpRequest
API, it will help you to understand how HTTP requests work under the hood.
Bundlers
In a typical web application, you can use javascript sources with <script src="..">
tags. However, if you want to use
libraries in your app, then you should import every library with <script src="..">
tags. Lets say you want use
CSR, SSR, SSG
Client side rendering is a technique that you render your HTML in browser. This means that you don't need to render your HTML in your server. This means that you don't need to use templating engines like EJS or handlebars.
Summary
Extra Resources
- HTML Wikipedia Page
- HTML in 100 Seconds
- CSS in 100 Seconds
- HTML in 5 minutes
- HTML Tutorial for Beginners: HTML Crash Course
Exercises
Security
Digital Signatures
Authentication
Authentication is the process of answering the question "Who are you?" Authentication: A process of determining whether someone or something is, in fact, who or what it is declared to be. It is the process of verifying the identity of a user or system. The goal of authentication is to establish the identity of the user or system requesting access to a resource.
Storing Passwords Securely
Multi-Factor Authentication
OAuth2
OAuth 2.0 is an authorization framework that is defined in RFC 6749. Its main goal is to allow a third-party application to obtain limited access to an HTTP service, either on behalf of a resource owner or by allowing the third-party application to obtain access on its own behalf.
In order to explain OAuth2, lets take an example. Imagine that you have an application that analyzes the tweets of a user and predicts the personality of the user. In order to analyze the tweets of the user, you need to access the tweets of the user. However, you cannot access the tweets of the user by directly calling the Twitter API. You need to get the permission of the user to access the tweets. This is where OAuth2 comes into play. By utilizing OAuth2, you can check your database whether you have a token for the user or not. If you do not have a token, you can redirect the user to the Twitter login page. This is the first step of the OAuth2 flow. OAuth2 describes the methods and parameters that are used to obtain and use access tokens to access protected resources. After the user logs in to the Twitter, Twitter will ask the user whether the user wants to give permission to your application to access the tweets. If the user accepts, Twitter will redirect the user back to your application.
This is the rough explanation of the OAuth2 flow. There are different flows that are used for different use cases. In this book, we will cover the most common flows such as Authorization Code Flow, Client Credentials Flow and Device Authorization Flow. We will not cover the Implicit Flow and Resource Owner Password Credentials Flow because they are not recommended by OAuth2 anymore.
Lets begin with OAuth2 roles:
- Resource Owner: An entity capable of granting access to a protected resource. When the resource owner is a person, it is referred to as an end-user.
- Resource Server: The server hosting the protected resources, capable of accepting and responding to protected resource requests using access tokens.
- Client: An application making protected resource requests on behalf of the resource owner and with its authorization
(e.g., a third-party application). There are two types of clients:
- Confidential: Clients capable of maintaining the confidentiality of their credentials (e.g., client ID and client secret).
- Public: Clients incapable of maintaining the confidentiality of their credentials (e.g., client ID and client secret).
- Authorization Server: The server that issues access tokens to the client after successfully authenticating the resource owner and obtaining authorization.
According to above explanation, the resource owner is the user, the resource server is the Twitter API, the client is your application and the authorization server is the Twitters authorization server. Lets look at different flows of OAuth2.
Authorization Code Flow (Recommended)
Authorization Code Flow with PKCE
Client Credentials Flow
Device Authorization Flow
Refresh Tokens
OAuth2 for Authentication
However, today OAuth2 is not only used for delegated authorization but also for authentication. This is a common misconception, but OAuth2 is not an authentication protocol. It is an authorization protocol that focuses on resource access and delegated authorization.
OpenID Connect
Authorization
Authorization is the process of answering the question "What are you allowed to do?". It is the process of determining whether a user or system has the necessary permissions to access a resource or perform a specific action. Authorization process starts after the authentication process. Once the user or system is authenticated, the authorization process determines what actions the user or system is allowed to perform.
Authorization may be a very complex process. In order to make everthing more clear, lets begin by defining some fundamental terms.
Access Control Matrix
Summary
Exercises
Extra Resources
Database
A database is an organized collection of data, generally stored and accessed electronically from a computer system. According to this definition, our file-based implementation is also a database.
Types of Databases
Query Language
A query language is a domain specific language used to make queries in databases and information systems (In this context context is a database). Complex queries can be expressed in a query language. There are many types of query languages.
SQL
SQL is the standard language for relational database management systems. SQL statements are used to perform tasks such as update data on a database, or retrieve data from a database.
In SQL, data is stored in tables. A table is a collection of related data entries and it consists of columns and rows.
SQL Injection Attack
SQL injection is a malicious technique in computer engineering where an attacker manipulates input data to insert or execute arbitrary SQL query within a web application's database. This vulnerability occurs when user inputs are not properly validated or sanitized, allowing unauthorized access to the database and potentially exposing sensitive data. SQL injection poses a significant security risk, as it can lead to data breaches, unauthorized data manipulation, and even full system compromise if not adequately mitigated. It is not only made by attackers but can occur by mistake by filling forms that runs SQL queries behind.
Programming Languages
Programming languages are used to describe the behavior of a computer (includes mobile phones, tablets, embedded devices and servers) to perform a specific task. There are many programming languages and each of them has its own strengths and weaknesses. Some of the programming languages are general purpose and some of them are domain specific. In this chapter, we will look different programming languages and their paradigms. We will try to classify them with different criteria such as typing, execution model, and paradigm.
A programming language is a formal specification of programs that can be executed by a computer. For this reason, it is important to know the difference between a programming language and implementation of a programming language.
Programming Language | Implementation(s) |
---|---|
C | GCC, Clang |
Java | OpenJDK, Oracle |
Python | CPython, PyPy |
Ruby | MRI, JRuby |
Javascript | V8, SpiderMonkey |
PHP | Zend Engine |
Go | gc, gccgo |
Rust | rustc |
Swift | Swift |
The syntax of a programming language describes the valid "sentences" that can be written in the language. On the other hand, the semantics of a programming language describes the meaning of the "sentences" that can be written in the language.
Programming languages often categorized by their paradigms, execution model and typing.
Compiled and Interpreted Languages
Most programming languages can be categorized as compiled or interpreted.
Programming Language | Interpreted | Compiled |
---|---|---|
Javascript | Yes | No |
Typescript | Yes | Yes |
Python | Yes | No |
Java | No | Yes |
C# | No | Yes |
PHP | Yes | No |
Go | No | Yes |
Imperative Programming Languages
Declarative Programming Languages
Summary
Exercises
Extra Resources
Type Safety
Programming Language | Typing |
---|---|
Typescript | Static |
Python | Dynamic |
Java | Static |
C# | Static |
PHP | Dynamic |
Go | Static |
Base Types
Function Types
Generics
Pairs, Tuples and Records
Sums and Variants
Algebraic Data Types
Enums
Subtyping
Recursive Types
Union and Intersection Types
Interfaces
Type Ascriptions
- Typescript Language Fundamentals
- Primitive Types (number, string, boolean, null, undefined)
- Type Assertions (as)
Data Structures
Primitive Data Types
Linear Data Structures
Trees
Graphs
Hash Tables
Serialization Formats: JSON
JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. You can use JSON to store your data or send your data over the network. JSON is a subset of JavaScript. It is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages. These properties make JSON an ideal data-interchange language.
Serialization Formats: XML
Serialization Formats: YAML
Summary
Exercises
Extra Resources
Object Oriented Programming
Object orient programming is a programming paradigm that organizes code into objects that combines data and behavior in a single unit. Commonly a unit of data stored in an object is called attribute and the a unit of behavior is called method. Object oriented programming is one of the most popular programming paradigms in the world. Most modern programming languages support object oriented programming today.
Why we need to combine data and behavior in a single unit? Lets examine a simple counter example. Suppose that we want to write a program that counts the number of times a button is clicked. We want to print the number to the console when a user clicks a button. We can write a simple program like this:
// Create a simple variable to store the count
var count = 0;
// Create a function to increment the count
function inc() {
// Increment the counter
count = count + 1;
// Print the counter to the console
console.log(count);
}
// Create a button element
// If you do not know javascript, just know that this block of triggers
// the inc function when the button is clicked
var button = document.createElement('button');
button.textContent = 'Click me';
button.onclick = () => inc();
document.body.appendChild(button);
This program will work as expected. However there are some problems with this program. The first one is that the
count
variable is a global variable. This means that any part of the program can change the value of the count
variable. This can lead to bugs that are hard to find. The second one is related to creating multiple counters.
If we want to create multiple counters, we need to create multiple count
variables and inc
functions. This
can lead to code duplication and hard to maintain code.
// Create a simple variable to store the count1
var count1 = 0;
// Create a function to increment the count1
function inc1() {
// Increment the counter
count1 = count1 + 1;
// Print the counter to the console
console.log(count1);
}
// Create a button element
// If you do not know javascript, just know that this block of triggers
// the inc1 function when the button is clicked
var button1 = document.createElement('button');
button1.textContent = 'Click me';
button1.onclick = () => inc1();
document.body.appendChild(button1);
// Create a simple variable to store the count2
var count2 = 0;
// Create a function to increment the count2
function inc2() {
// Increment the counter
count2 = count2 + 1;
// Print the counter to the console
console.log(count2);
}
// Create a button element
// If you do not know javascript, just know that this block of triggers
// the inc2 function when the button is clicked
var button2 = document.createElement('button');
button2.textContent = 'Click me';
button2.onclick = () => inc2();
document.body.appendChild(button2);
This code is too dirty and hard to maintain. Object oriented programming solves these problems by introducing a method to combine data and behavior in a single unit. This unit is called an object. Lets rewrite the counter example using objects.
// Create counter1 object
var counter1 = {
count: 0, // <-- Attribute
inc: function () {
counter1.count = counter1.count + 1;
console.log(counter1.count);
}, // <-- Method
};
// Create counter2 object
var counter2 = {
count: 0, // <-- Attribute
inc: function () {
counter1.count = counter1.count + 1;
console.log(counter1.count);
}, // <-- Method
};
// Create a button element
var button1 = document.createElement('button');
button1.textContent = 'Click me';
button1.onclick = () => counter1.inc();
document.body.appendChild(button1);
// Create a button element
var button2 = document.createElement('button');
button2.textContent = 'Click me';
button2.onclick = () => counter2.inc();
document.body.appendChild(button2);
This code is much cleaner compared to previos one. However as you can notice, the inc
method contains the same code
except the object name. In order to remove code duplication, we can receive the object the we interested in as a parameter
to the inc
method.
// Create counter1 object
var counter1 = {
count: 0, // <-- Attribute
};
// Create counter2 object
var counter2 = {
count: 0, // <-- Attribute
};
function inc(counter) {
// <-- Receive the object as a parameter (again it is a method)
counter.count = counter.count + 1;
console.log(counter.count);
}
// Create a button element
var button1 = document.createElement('button');
button1.textContent = 'Click me';
button1.onclick = () => inc(counter1); // <-- Pass counter1 object to the inc function
document.body.appendChild(button1);
// Create a button element
var button2 = document.createElement('button');
button2.textContent = 'Click me';
button2.onclick = () => inc(counter2); // <-- Pass counter2 object to the inc function
document.body.appendChild(button2);
By passing the object to the inc
method, we removed the code duplication. This is the basic idea of object oriented
programming. Instead of passing the object that we interested in as an explicit parameter, we can use a special keyword
called this
which is passed to the function implicitly. Lets rewrite the counter example using this
keyword.
var counter1 = {
count: 0,
inc: function () {
this.count = this.count + 1;
console.log(this.count);
},
};
var counter2 = {
count: 0,
inc: function () {
this.count = this.count + 1;
console.log(this.count);
},
};
var button1 = document.createElement('button');
button1.textContent = 'Click me';
button1.onclick = () => counter1.inc(); // <-- No need to pass the object explicitly
document.body.appendChild(button1);
var button2 = document.createElement('button');
button2.textContent = 'Click me';
button2.onclick = () => counter2.inc(); // <-- No need to pass the object explicitly
document.body.appendChild(button2);
However instead of specifying each attribute's initial value one by one we can use a function that will create the object for us. This function is called a constructor. Lets rewrite the counter example using a constructor.
function Counter() {
// <-- Constructor function
return {
count: 0,
inc: function () {
this.count = this.count + 1;
console.log(this.count);
},
};
}
var counter1 = Counter(); // <-- Create counter1 object
var counter2 = Counter(); // <-- Create counter2 object
var button1 = document.createElement('button');
button1.textContent = 'Click me';
button1.onclick = () => counter1.inc();
document.body.appendChild(button1);
var button2 = document.createElement('button');
button2.textContent = 'Click me';
button2.onclick = () => counter2.inc();
document.body.appendChild(button2);
This example shows the basic idea of object oriented programming and covers attribute
, method
, and constructor
concepts. Although there are some improvements that can be made, this example is enough to understand the basic idea
of object oriented programming and we will cover these improvements in the following sections.
Classes
A class is a blueprint for an object. It defines common attributes and methods that objects created from the class will have. In the previous examples, we created objects using object literals and constructor functions. However modern programming languages provide a special syntax to define classes. Lets rewrite the counter example using classes.
class Counter {
constructor() {
this.count = 0;
}
inc() {
this.count = this.count + 1;
console.log(this.count);
}
}
var counter1 = new Counter();
var counter2 = new Counter();
A class is a type in a typed programming language. This means that we can create a variable that has a type of a class. In the previous examples, we created objects using object literals and constructor functions. Lets examine an example for Java programming language.
class Counter {
int count = 0;
void inc() {
count = count + 1;
System.out.println(count);
}
}
public class Main {
public static void main(String[] args) {
Counter counter1 = new Counter(); // <-- Type of counter1 is Counter
Counter counter2 = new Counter(); // <-- Type of counter2 is Counter
}
}
Therefore, you should notice that a class is not a concrete object. It is just a blueprint for an object. When you create
an instance of a class, it becomes an object. In the previous examples, Counter
is a class and counter1
and counter2
are objects.
Encapsulation
Encapsulation is a concept that restricts direct access to some of the object's attrubutes and methods. Encapsulation is used to protect the object's state from being modified by other objects. In object oriented programming, we can restrict access to an object's attributes and methods by using access modifiers.
For instance, in our counter example, the count
attribute is public which means that any part of the program can
change the value of the count
attribute. This can lead to bugs that are hard to find. We can restrict access to the
count
attribute by using access modifiers. Access modifiers are language specific and they can be different from
language to language. Lets examine an example for Typescript programming language.
class Counter {
private count = 0; // <-- Private access modifier
private printToConsole() {
// <-- Private access modifier
console.log(this.count);
}
inc() {
// <-- Public access modifier by default
this.count = this.count + 1; // <-- Access to the private attribute
this.printToConsole(); // <-- Access to the private method
}
}
var counter1 = new Counter();
var counter2 = new Counter();
counter1.inc(); // <-- Access to the public method
counter2.inc(); // <-- Access to the public method
In this example, the count
attribute is private which means that only the methods of the Counter
class can access
the count
attribute directly. The printToConsole
method is also private which means that only the methods of the
Counter
class can access the printToConsole
method directly. The inc
method is public by default which means
that any part of the program can access the inc
method.
Inheritance
Inheritance is a concept that allows a class to inherit attributes and methods from another class. The class that inherits attributes and methods is called a subclass and the class that provides attributes and methods is called a superclass. Lets examine an example for Typescript programming language.
class Counter {
private count = 0;
private printToConsole() {
console.log(this.count);
}
inc() {
this.count = this.count + 1;
this.printToConsole();
}
}
class ResettableCounter extends Counter {
reset() {
this.count = 0;
this.printToConsole();
}
}
var counter1 = new ResettableCounter();
var counter2 = new Counter();
counter1.inc(); // <-- Access to the public method of the Counter class
counter1.reset(); // <-- Access to the public method of the ResettableCounter class
counter2.inc(); // <-- Access to the public method of the Counter class
In this example, the ResettableCounter
class is a subclass of the Counter
class. The ResettableCounter
class
inherits the count
attribute and the printToConsole
method from the Counter
class. The ResettableCounter
class
also has a reset
method that is not available in the Counter
class. The ResettableCounter
class can access the
count
attribute and the printToConsole
method directly. The ResettableCounter
class can also access the inc
method of the Counter
class.
Also, it is possible to override the methods of the superclass in the subclass. Lets examine an example for Typescript programming language.
class Counter {
private count = 0;
private printToConsole() {
console.log(this.count);
}
inc() {
this.count = this.count + 1;
this.printToConsole();
}
}
class ResettableCounter extends Counter {
private printToConsole() {
console.log('ResettableCounter printToConsole method');
console.log(this.count);
}
reset() {
this.count = 0;
this.printToConsole();
}
}
var counter1 = new ResettableCounter();
var counter2 = new Counter();
counter1.inc();
counter1.reset();
counter1.inc();
counter2.inc();
Therefore, by using inheritance, we can remove code duplication and create a hierarchy of classes.
Polymorphism
Work in Progress
Abstraction
Abstraction is a concept that hides the implementation details of an object and only shows the necessary details to the outside world. Abstraction is used to simplify the complexity of an object. In object oriented programming, we can use abstract classes and interfaces to achieve abstraction.
An interface is a specification of a class. It defines a set of methods that a class must implement. Lets examine an example for Typescript programming language.
interface Counter {
count: number;
inc(): void;
}
class InMemoryCounter implements Counter {
count = 0;
inc() {
this.count = this.count + 1;
console.log(this.count);
}
}
class LocalStorageCounter implements Counter {
count = 0;
constructor() {
this.count = parseInt(localStorage.getItem('count') || '0');
}
inc() {
this.count = this.count + 1;
localStorage.setItem('count', this.count.toString());
console.log(this.count);
}
}
// Gets an instance of the Counter interface and increments the counter n times
function countNTimes(counter: Counter, n: number) {
for (var i = 0; i < n; i++) {
counter.inc(); // <-- We do not know which concrete class will be used here
}
}
var counter1 = new InMemoryCounter();
var counter2 = new LocalStorageCounter();
counter1.inc();
counter2.inc();
// Refresh the page and run the program again
When you refresh the page and run the program again, you will see that the count
attribute of the LocalStorageCounter
class is persisted in the local storage. This is an example of abstraction. The countNTimes
function does not know
which concrete class will be used. It only knows that the class will have a count
attribute and an inc
method.
It does not care about the implementation details of the class.
This way of calling a method without knowing the implementation details is called dynamic dispatch. Dynamic dispatch
is a mechanism that determines which method to call at runtime. In the previous example, the countNTimes
function
uses dynamic dispatch to call the inc
method of the Counter
interface. On the other hand, counter1.inc()
and
counter2.inc()
calls the inc
method of the InMemoryCounter
and LocalStorageCounter
classes respectively. This
is called static dispatch. Static dispatch is a mechanism that determines which method to call at compile time.
Prototype Chains (JavaScript Specific, Optional)
Generally, an object oriented programming language has two choices for implementing object oriented programming. The first one is class-based and the second one is prototype-based. In class-based object oriented programming, a class is a blueprint for an object and an object is an instance of a class.
Work in Progress
Summary
Exercises
Extra Resources
Functional Programming
Functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids changing state and mutable data. It is a declarative programming paradigm, which means that programming is done with expressions or declarations instead of statements.
Some programming languages support functional programming paradigm in addition to other paradigms. For instance, JavaScript supports functional programming paradigm in addition to object-oriented programming paradigm. On the other hand, some programming languages are designed to support only functional programming paradigm. For instance, Haskell is a purely functional programming language.
We will look at some of the core concepts of functional programming in this chapter.
Pure Functions
There is a difference between functions in mathematics and functions in programming. In mathematics, a function is a relation between a set of inputs and a set of possible outputs. Therefore, a function satisfies these two properties:
- A function should return the same output for the same input.
- A function should not have side effects.
However, in programming, a function can have side effects and it can return different outputs for the same input. For instance, a random number generator function is a function that returns different outputs for the same input. It does not take any arguments but it returns a different output each time you call it. (If it would be a function in a mathematical sense, it should return only one value because it does not take any arguments.).
A function that prints something to the screen is a function that has side effects. It does not return anything but it changes the state of the screen. Or a function that send a request to a server is a function that has side effects because it manipulates the "outside world" by using electrical signals. A function that reads a file from the disk is also a function that has side effects because it causes the disk to spin and read the file that is located on a specific location on the disk.
Lets look at some examples;
// Some pure functions
(x, y) => x + y;
(person) => (person.age > 18 ? 'Adult' : 'Child');
(str) => JSON.parse(str);
// Some impure functions
(x, y) => x + y + Math.random();
(x, y) => console.log(x + y);
() => fs.readFileSync('file.txt');
(x, y) => fs.writeFileSync('file.txt', x + y);
(x, y) => {
console.log(x + y);
return x + y;
};
In some functional programming languages, all functions must be pure functions (such as Haskell). Pure functions are very useful because they are easy to test. You can test a pure function by giving it an input and checking the output. If the output is the same as you expected, then the function is working correctly. Also, pure functions are easy to understand and most importantly, they are easy to compose. Therefore, building complex systems with pure functions is much easier than building them with impure functions. Because, they are guaranteed to work correctly for each individual function and composing them will not result any unexpected behavior.
First Class Functions
Immutablity
One of the most bug-prone parts of a code is mutable variables (or variables). Because, you can change the value of a
variable at any time from any place. This can cause a lot of bugs. In order to avoid these bugs, you should use immutable
variables as much as possible. Instead of using let
and var
, you should use const
in order to make your variables
immutable. Also, you should avoid mutating objects and arrays. You should use map
, filter
, reduce
functions to
manipulate arrays.
Also you should be careful about scoping. Because scopes can restrict the usage of a variable. If you are defining a variable in a global scope, you can use it from anywhere. This can cause a lot of bugs. For this reason, please try to use local variables as much as possible and keep your scopes as small as possible.
It may be seem to be impossible to avoid mutable variables. However, you can use some techniques to avoid them. For
instance, you can use Map, Filter and Reduce functions can be used. They are higher order functions which means that they
take a function as an argument and they return a new array. These functions are very useful and they are reduces the
need for loops and mutable variables. You should use these functions as much as possible. Also there are some other
functions in javascript that you can use to manipulate arrays. For instance, forEach
, some
, every
, find
, findIndex
etc. You should use these functions as much as possible. Also, you can use lodash
library to manipulate objects and
arrays.
Also, you should return results of a function with return
statement. Mutating a global variable and then reading it
from another function is a bad practice. Instead of that, you should return the result of a function with return
statement. This will make your code much more clear and easy to understand.
If you want to learn more about immutability, you can check Functional Programming Paradigm.
Common Higher Order Functions
Side Effects with Pure Functions
You may wonder that how is it possible to write a program only with pure functions. Because, in real world, you need to interact with the outside world. You need to read files, send requests to servers, print something to the screen etc. However, this is possible in Haskell. At the end of the day nothing is pure because the nature of the problem that you want to solve is not pure. Lets look at an example from Haskell.
-- You can run this program with `runhaskell` command in your terminal but you need to have ghc installed on your system.
-- putStrLn is a function that takes a string and returns an IO action that DESCRIBES how to print the given string to
-- the screen. Type signiture of putStrLn is:
-- putStrLn :: String -> IO ()
-- main is a DESCRIPTION of the program. It is an IO action that DESCRIBES how to run the program.
-- It is not a function it is just a value that describes how to run the program.
main:: IO ()
main = putStrLn "Hello, World!" -- In haskell, we call functions with space instead of parentheses.
You should think IO
as a recipe that describes what a computer should do. Although, executing instructions in a computer
is not a pure operation, the recipe that includes these instructions itself is a completely pure value. Lets look at
another example.
-- This is a pure function that takes a string and returns a string
greet :: String -> String
greet name = "Hello, " ++ name ++ "!"
main:: IO ()
main = putStrLn (greet "John")
Lets examine the code above. The haskell compiler first looks at the main
function. It sees that main
is an IO action
that describes how to run the program. However in order to get a complete list of instructions, it needs to evaluate the
putStrLn (greet "John")
expression. In order to evaluate this expression, it needs to evaluate the greet "John"
expression. Therefore it evaluates the greet
function with the argument "John"
. The greet
function returns a string
that is "Hello, John!"
. Then the compiler evaluates the putStrLn
function with the argument "Hello, John!"
. The
putStrLn
function returns an IO action that describes how to print the given string to the screen. We can see its like
this:
main = `Print "Hello, John!" to the screen`
After that, the compiler runs the instructions that are described in the main
function. It prints the string "Hello, John!"
to the screen. Lets look at a more complex example.
greet :: String -> String
greet name = "Hello, " ++ name ++ "!"
-- Lets write our main function that will ask the name from the user and greet the user
main :: IO ()
main = putStrLn "What is your name?"
Summary
Exercises
Extra Resources
Tools
Code Editors and IDEs
You can use VSCode as your IDE. It is a good IDE for Javascript and Node.js development. Vscode has a lot of plugins
that you can use. For instance, you can use Prettier
plugin to format your code. You can use ESLint
plugin to check
your code quality. You can use GitLens
plugin to see git history.
Visual studio supports sharing your IDE settings across different machines. You can use .vscode
folder to store your
settings. You can put your settings in settings.json
file. You can put your tasks in tasks.json
file. These are
files are very imprtant because working in projects with different settings can be a problem. So, you need to keep your
settings in version control. (Described in the next section)
Markdown
You will write your documentation in markdown format. Markdown is a lightweight markup language with plain text formatting syntax. It is designed so that it can be converted to HTML and many other formats using a tool. Markdown is often used to format README files, for writing messages in online discussion forums, and to create rich text using a plain text editor. You can check markdown format from here.
Package Managers
What is package manager? Package manager constraints Lock files Workspaces Monorepos
Programming Language | Canonical Package Manager |
---|---|
Javascript | npm |
Typescript | npm |
Python | pip |
Java | Maven, Gradle |
C# | NuGet |
PHP | Composer |
Go | Go Modules |
Formatters
A code can be seen unclear if it is not formatted correctly. Formatting covers spaces, tabs, new lines, semicolons etc. A code formatter is a tool that automatically formats your code. Formatting means changing the code style. For instance, you can use a code formatter to change
- all double quotes to single quotes
- all tabs to spaces
- all semicolons to no semicolons
or any other code style. A code formatter is a tool that you can use to format your code. It is a good practice to use a code formatter. It will help you to keep your code clean. However, please do not forget that you need to setup your code formatter correctly. Also, different code formatting rules across different projects can be a problem. So, you need to add a configuration file to your project in order to keep your code style consistent. Also, configuration file should be under version control. So, all developers can use the same code style.
Programming Language | Canonical Formatter |
---|---|
Javascript | Prettier |
Typescript | Prettier |
Python | Black |
Java | Google Java Format |
C# | dotnet format |
PHP | PHP-CS-Fixer |
Go | Goimports |
Linters
Programming Language | Canonical Linter |
---|---|
Javascript | ESLint |
Typescript | ESLint |
Python | Pylint |
Java | Checkstyle |
C# | StyleCop |
PHP | PHP_CodeSniffer |
Go | GolangCI-Lint |
Markdown | markdownlint |
Summary
Exercises
Extra Resources
Advices
Use Git Properly
Naming
Naming: You should be careful about naming. You should use meaningful names for your variables, functions, classes etc.
Comments
Your comments should be as short as possible. You should write comments to explain your code but in general your code should be self-explanatory. You should not write comments that are not needed. You can explain algorithms, data structures, design patterns etc. in your comments. You should not write comments that are explaining what your code does. Instead, your code should explain what it does in a clear way.
Nesting
You should avoid deeply nested code. You should use early returns to avoid nested code.
Command Line Interface Challenge
In this chapter, we will begin our development journey by implementing a simple command line interface (CLI) application. We will implement a TODO list application that can perform basic operations in this chapter and followings. We will improve our TODO list application step by step. You will see bad examples and good examples in each chapter. The main purpose of of this part of the book is to show how to improve your coding skills by using different techniques and technologies.
Requirements
We want to implement a simple TODO list application that works on the command line. Before starting to implement the application, we need to define the requirements of the application. Lets define our requirements in two different categories: functional and non-functional requirements.
A functional requirement describes what a software system should do. It specifies the behavior of the system. In our case, the functional requirements of the TODO list application might be like this:
- As a user, I should be able to list all of the todos that is created before.
- As a user, I should be able to add a new todo to the list.
- The todo should have a unique identifier.
- Title of the todo must be a string and it is required.
- Maximum length of the title is 255 characters.
- Minimum length of the title is 2 character.
- Description of the todo must be a string and it is optional.
- Maximum length of the description is 4096 characters.
- The todo should have a status, it is either done or undone.
- As a user, I should be able to mark a todo as done.
- As a user, I should be able to mark a todo as undone.
- As a user, I should be able to delete a todo from the list.
A non-functional requirement describes how a software system should do. It specifies more technical aspects of the system. In our case, the non-functional requirements of the TODO list application might be like this:
- The application should be implemented in TypeScript.
- The application should use asynchronous operations instead of synchronous (blocking) operations.
- As a user, I should be able to add a todo by giving this command:
tdr add "A sample task" "A sample task description"
. - As a user, I should be able to list all of the todos by giving this command:
tdr list
. - As a user, I should be able to mark a todo as done by giving this command:
tdr done id-of-todo
. - As a user, I should be able to mark a todo as undone by giving this command:
tdr undone id-of-todo
. - As a user, I should be able to delete a todo by giving this command:
tdr delete id-of-todo
. - The application should store all of the todos in a JSON file.
We are using JSON file to store the todos. This is not the best way to store the todos but we will use it for simplicity. In the following chapters, we will improve the storage mechanism of the todos. Here is a sample JSON file that stores a single todo:
[
{
"id": 1,
"title": "A sample task description",
"description": "This is a sample task description",
"done": false
}
]
Also, as a non-functional requirement we will use asynchronous file operations in this chapter1. We will use the fs
module of the Node.js to read and write the JSON file. The fs
module provides basic file operations in two different
ways: synchronous and asynchronous. For instance, fs.readFileSync
function is used to read a file synchronously and
fs.readFile
function is used to read a file asynchronously. When you use fs.readFileSync
function, the execution of
the program is blocked until the file is read. When you use fs.readFile
function, the execution of the program is not
blocked and the file is read in the background. Another variant of the fs
module is the fs.promises
module. The
fs.promises
module provides the same functions as the fs
module but they return promises instead of using callbacks.
We will use the fs.promises
module in this chapter.
Project Scaffolding
Each software project has a structure and each of them starts with a scaffolding. Scaffolding is a process of creating the initial structure of the project. I will not use any project scaffolding tools in this book and also I do not recommend you to use them. Because, they will hide the underlying concepts and technologies from you. Instead of that, I will create the initial structure of the project by hand and I will explain each step in detail. So, you can understand the underlying concepts and technologies in detail.
# Create a new directory for our project
mkdir tdr
# Change the directory to the project directory
cd tdr
# Initialize the git repository
git init
# Initialize the .gitignore file
curl https://www.toptal.com/developers/gitignore/api/linux,macos,windows,node > .gitignore
# Initialize the package.json file
npm init -y
# Create the src directory
mkdir src
touch src/index.ts
# Create a directory for Github Actions
mkdir .github
# Create a directory for VSCode settings
mkdir .vscode
touch .vscode/settings.json
# Initialize the code formatter
npm i -D prettier
touch .prettierrc
# Install the typescript and tsx (Typescript execute)
npm i -D typescript tsx @types/node
# Initialize the typescript configuration
npx tsc --init
# Create a README.md file
touch README.md
As you can see, the script above starts by creating a new directory named tdr
and then it changes the current directory
to the tdr
directory 2. After that, it initializes a git repository in the created directory3.
After initializing the git repository, we are adding a .gitignore
file to the project. Here, instead of creating
a .gitignore
file by hand, we are using a gitignore generator tool which is served by Toptal. As a reminder,
.gitignore
file is used to specify which files and directories should be ignored by git. These file name patterns are
generally does not change too much. For this reason, we can use a predefined .gitignore
file for our project. For this
reason, we are sending an HTTP request to the Toptal gitignore generator tool and we are saving the response to the
.gitignore
file. Please not that, these files start with a dot (.
) character which means that they are hidden files.
After that, we are creating a package.json
file by using the npm init -y
command. This command creates a
package.json
file with default values. We will update this file later. Simply package.json
file is used to store
metadata of the project such as project name, version, description, dependencies, and scripts.
After creating the package.json
file, we are creating a src
directory and an index.ts
file in it. We will use the
src
directory to store all of our source files and the index.ts
file will be the entry point of our application.4
After these operations, we are creating two empty directories named .vscode
and .github
. We will use the .vscode
directory to store the Visual Studio Code related configurations and the .github
directory to store the GitHub related
configurations. Lets add some configurations to the .vscode
directory in order to format our code automatically when
we save the file. We can add the following lines to the .vscode/settings.json
file:
{
"editor.formatOnSave": true,
"editor.formatOnPaste": true,
"editor.defaultFormatter": "esbenp.prettier-vscode"
}
These settings will be pushed to the GitHub repository and they will be used by the other developers who will clone the repository. This feature is very useful because it ensures that all of the developers use the same settings and the same tools. This way, we can create a consistent codebase.
A tool that we will use so much in this book is prettier
. Prettier is a code formatter tool that can format your code
automatically. In order to use the prettier
tool, we need to install it first. We are installing it as a development
dependency by giving -D
flag to the npm install
command4. After installing the prettier
tool, we are creating a
.prettierrc
file to store the configuration of the prettier
tool. In this file, we can specify the configuration of
the prettier
tool such as the maximum line width, the tab width, and the trailing comma. Lets add the following
configuration to the .prettierrc
file:
{
"tabWidth": 2,
"useTabs": false,
"semi": true,
"singleQuote": true,
"trailingComma": "all",
"bracketSpacing": true,
"bracketSameLine": true,
"arrowParens": "always",
"endOfLine": "auto"
}
Now we can install the tools for the TypeScript development. We need to install the typescript
package to compile our
TypeScript code to JavaScript code. Also, we need to install the tsx
package to run our TypeScript code. In addition to
these packages, we need to install the @types/node
package to use the Node.js types in our TypeScript code. @types/node
package contains the type definitions of the Node.js core modules.
We can create a tsconfig.json
file by running the npx tsc --init
command. This command creates a tsconfig.json
file
with default values. We will update this file later. tsconfig.json
file is used to store the configuration of the
TypeScript compiler. In this file, we can specify the target version of the JavaScript, the module system, the output
directory, and many other options.
As last step, we are creating a README.md
file to store the documentation of the project. We will not add any content
to this file in this book. But, you can add any content you want to this file. README.md
file is an important piece
of any project. It is the first point of contact for the users and the developers of the project. For this reason, please
try to write a good documentation for your project.
In order to define aliases for common commands, we can add the following lines to the scripts
field of the
package.json
:
"main": "src/index.ts",
"scripts": {
"format": "prettier --write .",
"format:check": "prettier --check .",
"start": "tsx src/index.ts",
"dev": "tsx --watch src/index.ts"
},
If you are not familiar with the async programming, please look at NOT READY
If you do not understand the script please look at Operating Systems Chapter
If you do not know what is git, please look at Version Control Chapter
If you are not familiar with the npm, please look at Programming Language Tools Chapter
Iteration 1. Implementing without any structure (a.k.a. spaghetti code)
The plan is simple:
- Get command line arguments.
- Check the existence of the todo list file. If it does not exist, create it.
- Parse the command line arguments and decide which operation to perform.
- Read the todo list file and parse it.
- Perform the operation.
- Write the updated list back to the file. (If it is necessary)
The first thing that we will need to do is getting arguments from the command line. We can use the process.argv
array
to get the command line arguments. The process.argv
array contains the command line arguments that are passed to the
Node.js process. The first element of the process.argv
array is the path to the Node.js executable and the second
element is the path to the JavaScript file that is executed. The rest of the elements are the command line arguments
that are passed to the Node.js process.
After getting the command line arguments, we can use fs.promises.stat
function to checks the existence of the
todo list file. If the file does not exist, we can create it by using the fs.promises.writeFile
function.
Then, we can simply use a switch-case statement to decide which operation to perform. In each case, we can execute different statements to perform the operation. I will not explain each line of the code in detail because instead it is an exercise for you.
Also in order to use async/await syntax, we need to define an async function. An async function is a function that
returns a promise, in other words, it is a function that produces a result which will be completed in the future. We
can use the async
keyword before the function definition to define an async function. We can use the await
keyword
inside the async function to wait for the completion of the promise. Node.js does not provide top-level await support
yet. This means that we cannot use the await
keyword outside of an async function. For this reason, we need to define
an async function and we need to call it in the top-level code. For this reason, we will define an async function named
main
and we will call it in the top-level code.
Lets implement the first version of our todo list application by adding the following lines to the src/index.ts
file:
#!/usr/bin/env -S npx tsx
import process from 'process';
import fs from 'fs';
import crypto from 'crypto';
// Define what a todo looks like
interface Todo {
id: string;
title: string;
description: string;
completed: boolean;
}
async function main() {
// process.argv is provided by Node.js and it is an array that contains the command line arguments
// The first element is the path to the Node.js executable
// The second element is the path to the script file
// The third element is the subcommand
// The rest of the elements are the arguments
// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;
// If the todo list file does not exist, create it
try {
await fs.promises.stat('todos.json');
} catch {
await fs.promises.writeFile('todos.json', '[]');
}
// Handle the subcommands
switch (subcommand) {
case 'add': {
const [title, description] = args;
// Read todo list file and parse it
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
// Validate the input
if (!title || typeof title !== 'string' || title.length < 2 || title.length > 255) {
console.log('Title is required and its length must be between 2 and 255 characters');
process.exit(1);
}
if (description && (typeof description !== 'string' || description.length > 4096)) {
console.log('Description length must be less than 4096 characters');
process.exit(1);
}
// Generate a new todo
const id = crypto.randomBytes(4).toString('hex');
const newTodo: Todo = {
id: id,
title,
description,
completed: false,
};
// Add the new todo to the list
todoList.push(newTodo);
// Write the updated list back to the file
const updatedFileContent = JSON.stringify(todoList, null, 2);
await fs.promises.writeFile('todos.json', updatedFileContent);
console.log('New todo added');
break;
}
case 'list': {
// Read todo list file and parse it
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
// Print the list of todos
for (const todo of todoList) {
const status = todo.completed ? 'X' : ' ';
console.log(`- [${status}] (id: ${todo.id}) ${todo.title}`);
if (todo.description) console.log(`\t${todo.description}`);
}
break;
}
case 'done': {
const [id] = args;
// Read todo list file and parse it
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
// Find the todo with the given id
const todo = todoList.find((todo) => todo.id === args[0]);
if (!todo) {
console.log('Todo not found');
process.exit(1);
}
// Mark the todo as completed
todo.completed = true;
// Write the updated list back to the file
const updatedFileContent = JSON.stringify(todoList, null, 2);
await fs.promises.writeFile('todos.json', updatedFileContent);
// Print the message
console.log('Todo marked as done');
break;
}
case 'undone': {
const [id] = args;
// Read todo list file and parse it
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
// Find the todo with the given id
const todo = todoList.find((todo) => todo.id === args[0]);
if (!todo) {
console.log('Todo not found');
process.exit(1);
}
// Mark the todo as not completed
todo.completed = false;
// Write the updated list back to the file
const updatedFileContent = JSON.stringify(todoList, null, 2);
await fs.promises.writeFile('todos.json', updatedFileContent);
// Print the message
console.log('Todo marked as undone');
break;
}
case 'delete': {
const [id] = args;
// Read todo list file and parse it
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
// Delete the todo with the given id
const index = todoList.findIndex((todo) => todo.id === args[0]);
if (index === -1) {
console.log('Todo not found');
process.exit(1);
}
todoList.splice(index, 1);
// Write the updated list back to the file
const updatedFileContent = JSON.stringify(todoList, null, 2);
await fs.promises.writeFile('todos.json', updatedFileContent);
// Print the message
console.log('Todo deleted');
break;
}
default:
// Print help messages
console.log(`Unknown subcommand`);
console.log(`Usage: tdr <subcommand> [args]`);
console.log(`Subcommands: add, list, done, undone, delete`);
// Exit with an error code
process.exit(1);
}
}
main();
And we can test the application by running the following commands:
# Add a new todo
npm run start add "A sample task description" "This is a sample task description"
npm run start add "Another task" "This is another task"
# List the todos
npm run start list
# Mark a todo as done
npm run start done #PutYourTodoIdHere#
# Mark a todo as undone
npm run start undone #PutYourTodoIdHere#
# Delete a todo
npm run start delete #PutYourTodoIdHere#
As you can see we are using npm run start
command to run the application. This command runs the start
script that is
defined in the package.json
file. The start
script runs the tsx src/index.ts
command. In order to run our script,
with a tdr
command, we need to define a bin
field in the package.json
file. Lets add the following lines to the
package.json
file:
"bin": {
"tdr": "src/index.ts"
},
This part of the code tells the npm that this package provides a command line utility named tdr
and it is located in
the src/index.ts
file. After adding this line executing npm install
command will create a directory under the
node_modules
directory and it will create a symbolic link to the src/index.ts
file. This way, we can run the
tdr
command from the command line by using the npx tdr
command. We need to use the npx
command because the tdr
command is not installed globally and therefore thh node_modules/.bin
directory is not in the PATH
environment
variable. The npx
command is used to run the commands that are located in the node_modules/.bin
directory. Lets test
the application by running npx tdr list
command.
Also, the first line of the src/index.ts
file is a shebang line. A shebang line is a special line that is used to
specify the interpreter that should be used to run the script. In our case, we are using tsx to run the TypeScript code.
For this reason, we are using the #!/usr/bin/env -S npx tsx
shebang line. This line tells the operating system to use
the provided command to run the script. We must provide shebang line in order to run the script directly from the command
line because operating system does not know how to run a script instead it is only capable of running binary files which
contains machine code for the processor. By providing a shebang line, actually we are delegating the execution of the
script to the provided command. In our case, we are delegating the execution of the script to the tsx
command.
Right now, we have a "working" todo list application but this is not the best implementation. Lets improve it.
Iteration 2. Implementing with functions and classes (semi-structured code)
In the previous implementation, we did not utilize functions. Without changing the requirements and the file structure of the todo list, lets refactor the code to use functions.
Functions are the building blocks of the software. They are used to encapsulate the logic and to make the code more
readable and maintainable. In our case, we can define different functions for each operation. For example, we can define
a listTodos
function to list all of the todos, a addTodo
function to add a new todo, a markTodoAsDone
function to
mark a todo as done, a markTodoAsUndone
function to mark a todo as undone, and a deleteTodo
function to delete a todo.
Also we can separate the functions to different files. This way, we can create a more modular and maintainable codebase.
It is a good way to group related functions in the same file. This way, we can easily find the related functions and
we can easily reuse them in different parts of the codebase. For this reason, we will use two different files: index.ts
and todo.ts
. We will move the todo related functions to the todo.ts
file and we will import them in the index.ts
#!/usr/bin/env -S npx tsx
import process from 'process';
import * as todoService from './TodoService';
async function main() {
// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;
switch (subcommand) {
case 'add': {
const [title, description] = args;
await todoService.addTodo(title, description);
}
case 'list': {
await todoService.listTodos();
break;
}
case 'done': {
const [id] = args;
await todoService.markTodoAsDone(id);
break;
}
case 'undone': {
const [id] = args;
await todoService.markTodoAsUndone(id);
break;
}
case 'delete': {
const [id] = args;
await todoService.deleteTodo(id);
break;
}
default:
// Print help messages
console.log(`Unknown subcommand`);
console.log(`Usage: tdr <subcommand> [args]`);
console.log(`Subcommands: add, list, done, undone, delete`);
// Exit with an error code
process.exit(1);
}
}
main();
As you can see, by separating the functions to different files, our index.ts
file becomes much more readable. Also,
its responsibility is reduced. It is now only responsible for parsing the command line arguments and calling the related
functions. Responsibility, this is the most important thing that you need to understand in this chapter.
When you separate your code into different pieces, you should ask this questions to yourself
What is the responsibility of this file (or this organization unit / module)?
If a file carries more than one responsibility, you should try to split responsibilities into different files. This way,
you can create a more maintainable and readable codebase. Therefore as a rule of thumb;
Single Responsibility Principle (SRP)
A class, function or module should have only one reason to change.
Lets look at TodoService.ts
file:
import fs from 'fs';
import crypto from 'crypto';
// Define what a todo looks like
export interface Todo {
id: string;
title: string;
description: string;
completed: boolean;
}
// Read todo list file and parse it
async function readTodos(): Promise<Todo[]> {
try {
await fs.promises.stat('todos.json');
} catch {
await fs.promises.writeFile('todos.json', '[]');
}
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
return todoList;
}
// Write the updated list back to the file
async function writeTodos(todoList: Todo[]): Promise<void> {
const updatedFileContent = JSON.stringify(todoList, null, 2);
await fs.promises.writeFile('todos.json', updatedFileContent);
}
export async function addTodo(title: string, description: string): Promise<void> {
const todoList = await readTodos();
// Validate the input
if (!title || typeof title !== 'string' || title.length < 2 || title.length > 255) {
console.log('Title is required and its length must be between 2 and 255 characters');
process.exit(1);
}
if (description && (typeof description !== 'string' || description.length > 4096)) {
console.log('Description length must be less than 4096 characters');
process.exit(1);
}
// Generate a new todo
const id = crypto.randomBytes(4).toString('hex');
const newTodo: Todo = {
id: id,
title,
description,
completed: false,
};
// Add the new todo to the list
todoList.push(newTodo);
writeTodos(todoList);
console.log('New todo added');
}
export async function listTodos() {
const todoList = await readTodos();
// Print the list of todos
for (const todo of todoList) {
const status = todo.completed ? 'X' : ' ';
console.log(`- [${status}] (id: ${todo.id}) ${todo.title}`);
if (todo.description) console.log(`\t${todo.description}`);
}
}
export async function markTodoAsDone(id: string) {
const todoList = await readTodos();
// Find the todo with the given id
const todo = todoList.find((todo) => todo.id === id);
if (!todo) {
console.log('Todo not found');
process.exit(1);
}
// Mark the todo as completed
todo.completed = true;
writeTodos(todoList);
// Print the message
console.log('Todo marked as done');
}
export async function markTodoAsUndone(id: string) {
const todoList = await readTodos();
// Find the todo with the given id
const todo = todoList.find((todo) => todo.id === id);
if (!todo) {
console.log('Todo not found');
process.exit(1);
}
// Mark the todo as not completed
todo.completed = false;
writeTodos(todoList);
// Print the message
console.log('Todo marked as undone');
}
export async function deleteTodo(id: string) {
const todoList = await readTodos();
// Delete the todo with the given id
const index = todoList.findIndex((todo) => todo.id === id);
if (index === -1) {
console.log('Todo not found');
process.exit(1);
}
todoList.splice(index, 1);
writeTodos(todoList);
// Print the message
console.log('Todo deleted');
}
As we can group related functions in the same file, we can also group related functions in a class (Although, there is
no so much difference between a class and a module in this example). Classes may be used to group related functions.
Actually classes are more than that. They are used to group related data and functions. In our case, we can define a
TodoService
class to group the todo related functions. Right now, we will not store any data in the TodoService
instances. However, we will add a state to the TodoService
class in the next chapter. For this reason, we will
use classes to group related functions. Let me show the code:
#!/usr/bin/env -S npx tsx
import process from 'process';
import TodoService from './TodoService';
const todoService = new TodoService();
async function main() {
// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;
switch (subcommand) {
case 'add': {
const [title, description] = args;
await todoService.addTodo(title, description);
}
case 'list': {
await todoService.listTodos();
break;
}
case 'done': {
const [id] = args;
await todoService.markTodoAsDone(id);
break;
}
case 'undone': {
const [id] = args;
await todoService.markTodoAsUndone(id);
break;
}
case 'delete': {
const [id] = args;
await todoService.deleteTodo(id);
break;
}
default:
// Print help messages
console.log(`Unknown subcommand`);
console.log(`Usage: tdr <subcommand> [args]`);
console.log(`Subcommands: add, list, done, undone, delete`);
// Exit with an error code
process.exit(1);
}
}
main();
import fs from 'fs';
import crypto from 'crypto';
// Define what a todo looks like
export interface Todo {
id: string;
title: string;
description: string;
completed: boolean;
}
// Define the TodoService class
class TodoService {
fileName = 'todos.json';
constructor() {}
private async readTodos(): Promise<Todo[]> {
try {
await fs.promises.stat('todos.json');
} catch {
await fs.promises.writeFile('todos.json', '[]');
}
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
return todoList;
}
private async writeTodos(todoList: Todo[]): Promise<void> {
const updatedFileContent = JSON.stringify(todoList, null, 2);
await fs.promises.writeFile(this.fileName, updatedFileContent);
}
public async addTodo(title: string, description: string): Promise<void> {
const todoList = await this.readTodos();
// Validate the input
if (!title || typeof title !== 'string' || title.length < 2 || title.length > 255) {
console.log('Title is required and its length must be between 2 and 255 characters');
process.exit(1);
}
if (description && (typeof description !== 'string' || description.length > 4096)) {
console.log('Description length must be less than 4096 characters');
process.exit(1);
}
// Generate a new todo
const id = crypto.randomBytes(4).toString('hex');
const newTodo: Todo = {
id: id,
title,
description,
completed: false,
};
// Add the new todo to the list
todoList.push(newTodo);
await this.writeTodos(todoList);
console.log('New todo added');
}
public async listTodos(): Promise<void> {
const todoList = await this.readTodos();
// Print the list of todos
for (const todo of todoList) {
const status = todo.completed ? 'X' : ' ';
console.log(`- [${status}] (id: ${todo.id}) ${todo.title}`);
if (todo.description) console.log(`\t${todo.description}`);
}
}
public async markTodoAsDone(id: string): Promise<void> {
const todoList = await this.readTodos();
// Find the todo with the given id
const todo = todoList.find((todo) => todo.id === id);
if (!todo) {
console.log('Todo not found');
process.exit(1);
}
// Mark the todo as completed
todo.completed = true;
await this.writeTodos(todoList);
// Print the message
console.log('Todo marked as done');
}
public async markTodoAsUndone(id: string): Promise<void> {
const todoList = await this.readTodos();
// Find the todo with the given id
const todo = todoList.find((todo) => todo.id === id);
if (!todo) {
console.log('Todo not found');
process.exit(1);
}
// Mark the todo as not completed
todo.completed = false;
await this.writeTodos(todoList);
// Print the message
console.log('Todo marked as undone');
}
public async deleteTodo(id: string): Promise<void> {
const todoList = await this.readTodos();
// Delete the todo with the given id
const index = todoList.findIndex((todo) => todo.id === id);
if (index === -1) {
console.log('Todo not found');
process.exit(1);
}
todoList.splice(index, 1);
await this.writeTodos(todoList);
// Print the message
console.log('Todo deleted');
}
}
export default TodoService;
Iteration 3. Layered Architecture
Right now, we have semi-structured code. It is semi-structured because it does not have a well-defined boundaries between the layers. In order to create more structured code, we need to define the parts of our application by providing a definition for each part. Lets try to define the parts of our application:
index.ts
: This is the entry point of our application. It is responsible for parsing the command line arguments and calling the correct functions or methods.TodoService.ts
: This is the service layer of our application. It is responsible for the business logic of the todo list. It is responsible for adding, listing, marking as done, marking as undone, and deleting the todos.
According to these definitions, we can say that TodoService
does not depend on index.ts
but index.ts
depends on
TodoService
. Therefore, we have two parts and one part depends another while the dependent part is independent.
Let me give an anology for this. We can see our project as a cake and parts of the project as layers of the cake.
Dependent part index.ts
is located at the top of the independent part TodoService.ts
. If underlying part of the
cake layer is not ready yet, you cannot put the upper layer on it. Therefore, the existence of upper layer depends on
the existence of the underlying layer.
Therefore, our current implementation has two layers right now. The first layer is the index.ts
file and the second
layer is the TodoService.ts
file. A careful reader may notice that our TodoService
does not have a single
responsibility because it handles;
- The business logic of the todo list.
- The file operations (reading and writing the JSON file).
- Printing the information to the console.
And, this conflicts with the Single Responsibility Principle. Therefore, we need to redifine our layers and arrange our files according to the new layers. Lets try to redefine our layers in a more rigorous way:
- Presentation Layer: Handles all of the user interactions. (Parsing the command line arguments and printing the results)
- Service Layer: Validates and runs all the business logic of the application.
- Data Access Layer: Stores and retries the data that is produced by the service layer.
According to these definitions, lets arrange our files:
- Presentation Layer: We should handle all of the operations that is related to user activity in
index.ts
file. Therefore, we need to handle all command line related operations in theindex.ts
file. - Service Layer: We should handle all of the business logic of the todo list in the
TodoService.ts
file and we should only handle bussiness logic in this layer. Therefore, we need to move console logging operations to theindex.ts
file and data access related operations to the Data Access Layer. - Data Access Layer: We should handle all of the file operations in the
TodoRepository.ts
file and again we should only handle file operations in this layer. We call itTodoRepository
because by definition (non-techical) a repository is a place where things are stored.
Lets refactor them by starting from TodoRepository.ts
:
import fs from 'fs';
export interface Todo {
id: string;
title: string;
description: string;
completed: boolean;
}
class TodoRepository {
fileName: string = 'todos.json';
constructor() {}
public async readTodos(): Promise<Todo[]> {
try {
await fs.promises.stat('todos.json');
} catch {
await fs.promises.writeFile('todos.json', '[]');
}
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
return todoList;
}
public async writeTodos(todoList: Todo[]): Promise<void> {
const updatedFileContent = JSON.stringify(todoList, null, 2);
await fs.promises.writeFile(this.fileName, updatedFileContent);
}
}
export default TodoRepository;
Our repository is so simple. Lets move to the TodoService.ts
file:
import TodoRepository, { Todo } from './TodoRepository';
import crypto from 'crypto';
// Define the TodoService class
class TodoService {
repository: TodoRepository;
constructor() {
this.repository = new TodoRepository();
}
public async addTodo(title: string, description: string): Promise<void> {
const todoList = await this.repository.readTodos();
// Validate the input
if (!title || typeof title !== 'string' || title.length < 2 || title.length > 255) throw new Error('Title is required and its length must be between 2 and 255 characters');
if (description && (typeof description !== 'string' || description.length > 4096)) throw new Error('Description length must be less than 4096 characters');
// Generate a new todo
const id = crypto.randomBytes(4).toString('hex');
const newTodo: Todo = {
id: id,
title,
description,
completed: false,
};
// Add the new todo to the list
todoList.push(newTodo);
await this.repository.writeTodos(todoList);
}
public async listTodos(): Promise<Todo[]> {
const todoList = await this.repository.readTodos();
return todoList;
}
public async markTodoAsDone(id: string): Promise<void> {
const todoList = await this.repository.readTodos();
// Find the todo with the given id
const todo = todoList.find((todo) => todo.id === id);
if (!todo) throw new Error('Todo not found');
// Mark the todo as completed
todo.completed = true;
await this.repository.writeTodos(todoList);
}
public async markTodoAsUndone(id: string): Promise<void> {
const todoList = await this.repository.readTodos();
// Find the todo with the given id
const todo = todoList.find((todo) => todo.id === id);
if (!todo) throw new Error('Todo not found');
// Mark the todo as not completed
todo.completed = false;
await this.repository.writeTodos(todoList);
}
public async deleteTodo(id: string): Promise<void> {
const todoList = await this.repository.readTodos();
// Delete the todo with the given id
const index = todoList.findIndex((todo) => todo.id === id);
if (index === -1) throw new Error('Todo not found');
todoList.splice(index, 1);
await this.repository.writeTodos(todoList);
}
}
export default TodoService;
As you can see, we do not have any methods for file operations in the TodoService
class. Instead, we are creating
an instance of the TodoRepository
class and we are using it to read and write the JSON file. This way, we are
separating the file operations from the business logic. Also, another improvement that we made is removing the console
logging operations from the TodoService
class. Instead of printing the information to the console directly in the
TodoService
class, we are returning the necessary information needed to print to the console to the caller. This way,
our TodoService
class becomes much more cleaner and maintainable. Lets refactor the index.ts
file:
#!/usr/bin/env -S npx tsx
import process from 'process';
import TodoService from './TodoService';
const todoService = new TodoService();
async function main() {
// Get command line arguments
const [program, script, subcommand, ...args] = process.argv;
switch (subcommand) {
case 'add': {
const [title, description] = args;
await todoService.addTodo(title, description);
console.log('New todo added');
}
case 'list': {
const todoList = await todoService.listTodos();
// Print the list of todos
for (const todo of todoList) {
const status = todo.completed ? 'X' : ' ';
console.log(`- [${status}] (id: ${todo.id}) ${todo.title}`);
if (todo.description) console.log(`\t${todo.description}`);
}
break;
}
case 'done': {
try {
const [id] = args;
await todoService.markTodoAsDone(id);
console.log('Todo marked as done');
break;
} catch (error) {
if (error instanceof Error) console.log(error.message);
process.exit(1);
}
}
case 'undone': {
try {
const [id] = args;
await todoService.markTodoAsUndone(id);
console.log('Todo marked as undone');
break;
} catch (error) {
if (error instanceof Error) console.log(error.message);
process.exit(1);
}
}
case 'delete': {
try {
const [id] = args;
await todoService.deleteTodo(id);
console.log('Todo deleted');
break;
} catch (error) {
if (error instanceof Error) console.log(error.message);
process.exit(1);
}
}
default:
// Print help messages
console.log(`Unknown subcommand`);
console.log(`Usage: tdr <subcommand> [args]`);
console.log(`Subcommands: add, list, done, undone, delete`);
// Exit with an error code
process.exit(1);
}
}
main();
Summary
In this chapter, you should have learned;
- What is a command line interface application?
- What is a functional and non-functional requirement?
- What is layered architecture?
- What is the Single Responsibility Principle?
- How to create a new project from scratch
- How to implement a simple command line interface application
- How to define functional and non-functional requirements
- How to implement a simple TODO list application
Also, you should refresh your knowledge about;
- How to serialize and deserialize JSON data
- How to use functions and classes in TypeScript
- How to use promises in Node.js
There are two important things that you need to get from this chapter:
- Single Responsibility Principle: A class, function or module should have only one reason to change.
- Layered Architecture: A software architecture that divides the software into different layers. Each layer has a specific responsibility and each layer depends on the underlying layer.
Exercises
- Look at prettier configuration parameters and try to understand meaning of each parameter.
- Implement each version of the TODO list application by yourself.
Database
In the previous chapter, we have implemented a simple command line TODO app and learned what is a layered architecture. However, the data is stored in a single unstructured JSON file is not the best way to store data. It has a lot of drawbacks such as:
- Adding, updating and deleting a single TODO item requires reading the whole file and writing the whole file back. If we have a large file, this will be very slow.
- If process crashes while writing the file, the file might be corrupted.
- Querying the data requires reading the whole file.
- If file is too large fitting it into memory will be a problem.
- Writing to a file is an error-prone operation. (If the system crashes while writing, the file might be corrupted)
In order to overcome these problems, we may have two options:
- Implementing more sophisticated file operations.
- Using a database.
The first solution will be hard to implement. For this reason, we will use a database in this chapter. However, in preceding chapters, we will also implement a simple file-based database in order to grasp the basics of databases. Without changing our functional requirements, we will refactor our code to store TODO items in a database. We will use SQLite because of its simplicity.
Single Responsibility Principle Revisited
Fortunately, we have separated our data access logic from the rest of our application. Therefore, we can easily switch to a different database system without changing the rest of our application. This is the Single Responsibility Principle in action. We should have only one reason to change our code. We have 3 parts and we have a single change that we want to make. Therefore, according to the Single Responsibility Principle, we should only change one part of our code. Let's look at the parts of our application and see what we need to change.
- Presentation Layer: Changing database is not a thing that user sees or interacts with. Therefore, this layer should not be affected by the change.
- Business Logic Layer: This layer is responsible for the business logic of our application. Our business logic which is mostly described by functional requirements of the previous chapter does not include any change related to the database. Therefore, this layer also should not be affected by the change.
- Data Access Layer: This layer is responsible for the data access logic of our application. This is the layer that we need to change. We can reimplement this layer for SQLite.
It looks clean right? But unfortunately, it is not the case. We have a problem. Our business logic layer is not separated from the data access layer properly. In other words, it is coupled to the data access layer in a way. Lets look at the problematic part of our business logic layer:
public async addTodo(title: string, description: string): Promise<void> {
const todoList = await this.repository.readTodos();
...
todoList.push(newTodo);
await this.repository.writeTodos(todoList);
}
Do you see the problem? This service layer assumes that the data stored in the repository may be read completely and written completely. Lets assume that we have a database with millions of records. Reading and writing all of them at once is not a good idea. We should read and write only the necessary parts of the data. Therefore, we need to change our service layer to work with the data access layer in a more granular way. We need to change our service layer and data access layer in order to get decoupled layers. Let's start with the repository.
import fs from 'fs';
export interface Todo {
id: string;
title: string;
description: string;
completed: boolean;
}
class TodoRepository {
fileName: string = 'todos.json';
constructor() {}
// Lets make this method private
private async readTodos(): Promise<Todo[]> {
try {
await fs.promises.stat('todos.json');
} catch {
await fs.promises.writeFile('todos.json', '[]');
}
const fileConent = await fs.promises.readFile('todos.json', 'utf-8');
const todoList = JSON.parse(fileConent) as Todo[];
return todoList;
}
// Also this one
private async writeTodos(todoList: Todo[]): Promise<void> {
const updatedFileContent = JSON.stringify(todoList, null, 2);
await fs.promises.writeFile(this.fileName, updatedFileContent);
}
public async createTodo(todo: Todo): Promise<void> {
const todoList = await this.readTodos();
todoList.push(todo);
await this.writeTodos(todoList);
}
public async deleteTodo(id: string): Promise<void> {
const todoList = await this.readTodos();
const index = todoList.findIndex((todo) => todo.id === id);
if (index === -1) throw new Error('Todo not found');
todoList.splice(index, 1);
await this.writeTodos(todoList);
}
public async listTodos(): Promise<Todo[]> {
const todoList = await this.readTodos();
return todoList;
}
// Only update provided fields
public async updateTodo(id: string, todo: Partial<Todo>): Promise<void> {
const todoList = await this.readTodos();
const index = todoList.findIndex((todo) => todo.id === id);
if (index === -1) throw new Error('Todo not found');
todoList[index] = { ...todoList[index], ...todo };
await this.writeTodos(todoList);
}
}
export default TodoRepository;
Lets refactor TodoService
to work with the new TodoRepository
:
import TodoRepository, { Todo } from './TodoRepository';
import crypto from 'crypto';
// Define the TodoService class
class TodoService {
repository: TodoRepository;
constructor() {
this.repository = new TodoRepository();
}
public async addTodo(title: string, description: string): Promise<void> {
// Validate the input
if (!title || typeof title !== 'string' || title.length < 2 || title.length > 255) throw new Error('Title is required and its length must be between 2 and 255 characters');
if (description && (typeof description !== 'string' || description.length > 4096)) throw new Error('Description length must be less than 4096 characters');
// Generate a new todo
const id = crypto.randomBytes(4).toString('hex');
const newTodo: Todo = {
id: id,
title,
description,
completed: false,
};
await this.repository.createTodo(newTodo);
}
public async listTodos(): Promise<Todo[]> {
return await this.repository.listTodos();
}
public async markTodoAsDone(id: string): Promise<void> {
await this.repository.updateTodo(id, { completed: true });
}
public async markTodoAsUndone(id: string): Promise<void> {
await this.repository.updateTodo(id, { completed: false });
}
public async deleteTodo(id: string): Promise<void> {
await this.repository.deleteTodo(id);
}
}
export default TodoService;
Do you see the difference? Our TodoService
is almost the same as our functional requirements. Let me remind you the
functional requirements:
- As a user, I should be able to list all of the todos that is created before.
- As a user, I should be able to add a new todo to the list.
- The todo should have a unique identifier.
- Title of the todo must be a string and it is required.
- Maximum length of the title is 255 characters.
- Minimum length of the title is 2 character.
- Description of the todo must be a string and it is optional.
- Maximum length of the description is 4096 characters.
- The todo should have a status, it is either done or undone.
- As a user, I should be able to mark a todo as done.
- As a user, I should be able to mark a todo as undone.
- As a user, I should be able to delete a todo from the list.
Right now, we have completely separated our business logic from the data access logic. You should be very careful about this kind of leaks. You need to be sure that your code really follows the Single Responsibility Principle. If you are not sure, you can always ask yourself the following questions:
- If I need to change a part of my code, how many parts of my code should be changed ?
- What is the definition of the part that I am looking at?
- Does the definition of the part that I am looking match with the implementation of the part?
Implementing SQLite Repository
npm install sqlite sqlite 3
-- Create a TODO table
CREATE TABLE todos (id TEXT, title TEXT, description TEXT, done BOOLEAN);
-- We can drop a table by using DROP TABLE statement
DROP TABLE todos;
-- We can update the definition of a table by using ALTER TABLE statement
ALTER TABLE todos ADD COLUMN id INTEGER PRIMARY KEY AUTOINCREMENT;
-- We can insert data into a table by using INSERT INTO statement
INSERT INTO todos (title, description, done) VALUES ('A sample task', 'This is a sample task', 0);
-- We can select data from a table by using SELECT statement
SELECT * FROM todos;
SELECT id, title FROM todos;
SELECT id, title FROM todos WHERE done = 0;
SELECT id, title FROM todos WHERE done = 0 ORDER BY id DESC;
SELECT id, title FROM todos WHERE done = 0 ORDER BY id DESC LIMIT 1;
-- We can update data in a table by using UPDATE statement
UPDATE todos SET done = 1 WHERE id = 1;
-- We can delete data from a table by using DELETE statement
DELETE FROM todos WHERE id = 1;
-- We can delete all data from a table by using DELETE statement without WHERE clause
DELETE FROM todos;
Lets rename our TodoRepository.ts
file to TodoFSRepository.ts
and create a new file TodoSQLiteRepository.ts
with
the following content:
import * as sqlite from 'sqlite';
import sqlite3 from 'sqlite3';
export interface Todo {
id: string;
title: string;
description: string;
completed: boolean;
}
class TodoSQLiteRepository {
db: sqlite.Database | null = null;
constructor() {}
private async getDatabase() {
if (this.db) return this.db;
// Open the database
this.db = await sqlite.open({
filename: 'todos.db',
driver: sqlite3.Database,
});
// Create the table if it doesn't exist
await this.db.run('CREATE TABLE IF NOT EXISTS todos (id TEXT PRIMARY KEY, title TEXT, description TEXT, done BOOLEAN);');
// Return the database
return this.db;
}
public async createTodo(todo: Todo): Promise<void> {
const db = await this.getDatabase();
await db.run('INSERT INTO todos (id, title, description, done) VALUES (?, ?, ?, ?);', [todo.id, todo.title, todo.description, todo.completed]);
}
public async listTodos(): Promise<Todo[]> {
const db = await this.getDatabase();
return await db.all('SELECT * FROM todos;');
}
public async updateTodo(id: string, todo: Partial<Todo>): Promise<void> {
if (!this.db) throw new Error('Database not initialized');
if (!id) throw new Error('ID is required');
if (todo.description) await this.db.run('UPDATE todos SET description = ? WHERE id = ?;', [todo.description, id]);
if (todo.completed) await this.db.run('UPDATE todos SET done = ? WHERE id = ?;', [todo.completed, id]);
if (todo.title) await this.db.run('UPDATE todos SET title = ? WHERE id = ?;', [todo.title, id]);
}
public async deleteTodo(id: string): Promise<void> {
if (!this.db) throw new Error('Database not initialized');
await this.db.run('DELETE FROM todos WHERE id = ?;', [id]);
}
}
export default TodoSQLiteRepository;
Combining two repositories (Wrong way)
import process from 'process';
import TodoFSRepository, { Todo } from './TodoFSRepository';
import TodoSQLiteService from './TodoSQLiteRepository';
export { Todo } from './TodoFSRepository';
class TodoRepository {
fsRepository: TodoFSRepository | null = null;
sqliteRepository: TodoSQLiteService | null = null;
constructor() {
if (process.env.DB === 'sqlite') {
this.sqliteRepository = new TodoSQLiteService();
} else {
this.fsRepository = new TodoFSRepository();
}
}
public async createTodo(todo: Todo): Promise<void> {
if (this.sqliteRepository) {
return this.sqliteRepository.createTodo(todo);
}
if (this.fsRepository) {
return this.fsRepository.createTodo(todo);
}
throw new Error('No repository found');
}
public async deleteTodo(id: string): Promise<void> {
if (this.sqliteRepository) {
return this.sqliteRepository.deleteTodo(id);
}
if (this.fsRepository) {
return this.fsRepository.deleteTodo(id);
}
throw new Error('No repository found');
}
public async listTodos(): Promise<Todo[]> {
if (this.sqliteRepository) {
return this.sqliteRepository.listTodos();
}
if (this.fsRepository) {
return this.fsRepository.listTodos();
}
throw new Error('No repository found');
}
public async updateTodo(id: string, todo: Partial<Todo>): Promise<void> {
if (this.sqliteRepository) {
return this.sqliteRepository.updateTodo(id, todo);
}
if (this.fsRepository) {
return this.fsRepository.updateTodo(id, todo);
}
throw new Error('No repository found');
}
}
export default TodoRepository;
Combining two repositories (Right way)
Summary
Exercises
- Implement an
InMemoryRepository
adapter that implements theRepository
interface. This adapter should store the data in memory.
Server Side Rendering
Right now, you have a working CLI based TODO app. However you noticed that sometimes you need to access these tasks from your mobile phone or your tablet. In order to do that you need to be able to access your computer from different devices. This means that you need a some kind of server that will be always be running and accessible from the internet. In this challange, you will implement a server side rendered TODO app. This means that you will have a server that will be running on a remote machine and you will be able to access it from your mobile phone, tablet or any other device that has a browser. You will be able to add, delete, update and list your tasks from your browser.
HTTP Server
In order to serve your web page, you need a HTTP server. A HTTP server is a program that listens for HTTP requests and
sends HTTP responses. There are a lot of HTTP servers. We will use express.js
in this challange. Express.js is a
framework that is built on top of node.js
. A sample express.js server:
const express = require('express');
const app = express();
app.get('/', (req, res) => {
res.send('<h1>Hello World</h1>');
});
app.listen(3000, () => {
console.log('Server is listening on port 3000');
});
Html Forms
An HTML Form (form
) is a simple way to collect input from user. Most commonly input is sent to a server to be
processed. form
element supports common input UI component types such as text
, checkbox
, radio
. Then the values
written by the user can be submitted to a server with the submit
element.
Template Engine
In order to create dynamic web pages, you need to use a template engine. A template engine is a program that is used to
create HTML documents. It takes a template and some data and creates an HTML document according to the template. There
are a lot of template engines. We will use handlebars
in this challange. A sample handlebars template:
<h1>{%title%}</h1>
<ul>
{%#each tasks%}
<li>{%this%}</li>
{%/each%}
</ul>
XSS Attack
Cross-Site Scripting (XSS) is a security vulnerability in web applications where untrusted data is improperly included in web page content, enabling malicious code execution in the context of other users' web browsers. This vulnerability occurs when web applications fail to adequately validate or sanitize input data, allowing an attacker to inject and execute code that can steal information, manipulate user sessions, or perform unauthorized actions on the affected website.
Implementing Controller-Service-Repository Pattern
Implementing with a template engine
Testing
Unit Testing with Jest
Integration Testing with Jest
Functional Testing with Cypress
Summary
Exercises
API Design
Implementing REST APIs
Implementing Server Sent Events (SSE)
Implementing WebSockets
Implementing gRPC
Summary
Exercises
Client Side Rendering
Right now, you have a working server side rendered TODO app. Afterawhile, you have recognized that you need a native mobile app for your TODO app. Lets assume that this mobile app is written by another developer. However, he/she needs to access your tasks. In order to do that, you need to create a REST API. This means that you need to communicate via JSON objects. In this challange, you will implement a REST API for your TODO app. Also, you will remove SSR support from your TODO app. This means that you will create a client side rendered TODO app.
In addition to these, also you noticed that you need to attach some files or photos to your tasks. In order to do that, you will use AWS S3. This means that you will upload files to S3 and store their URLs in your database.
In this challenge, it is expected to learn basics of:
- AWS S3 (with presigned URLs)
- File Upload from browser
This challange does not introduce any new feature other than uploading files to S3.
Requirements
You can use libraries like axios and lodash. Also in the backend side, there are no restrictions. You can use any library you want. However, you should not use any React like framework or library. The purpose of this challenge is to learn the basics of client side rendering by using vanilla Javascript.
File Upload and S3
Sending files over HTTP is a common need in almost every web application. In this challange, you will try to upload files to S3. S3 is a cloud storage service provided by AWS. You can upload files to S3 and download them later. We will not use FTP because it is not a good practice to store files in your server. If you would use FTP, horizontal scaling would be a problem. Also, you would need to store files in your server. This means that you would need to pay for storage. However, S3 is a cloud storage service. This means that you will only pay for the storage that you use. Also, you can scale your application easily.
There are two approaches to upload files to S3. The first one is to upload files directly from your backend. This means that your backend will receive a file from browser and upload it to S3. The second approach is to upload files directly from browser. This means that your browser will upload a file to S3. The second approach is better because you don't need to send files to your backend. This means that you don't need to pay for bandwidth.
S3 Presigned Url
If you have an application where users need to upload files to your S3 bucket, you can provide them with a presigned URL to upload a file directly, without them having AWS access. This way you do not need to share AWS credentials or make the file public. The presigned URL is generated by server that has AWS credentials and permission to access the object. The URL embeds the permissions as query string parameters and a signature to ensure they can't be tampered with. The URL is valid for a limited time, defined when the URL is created. After this time, the URL will no longer provide access to the object.
Implementing with Vanilla JavaScript
Reactive Programming
Client side rendering has gained popularity in the last decade due to the rise of browser-based applications. In server side rendering, browser do not a heavy lifting, it just renders the HTML sent by the server. In client side rendering, the browser has to do a lot of work to render the page. For this reason, it was unconvenient to use client side rendering in the past.
However, managing a complicated client side rendering with traditional server-side rendering methods is not practical. This is the problem most frontend frameworks try to solve. React is one of the most popular frontend frameworks that solves this problem. In this challenge, it is expected to learn these concepts and apply them in a project:
Implementing with React
Summary
Exercises
Distributed Systems
ACID
CAP Theorem
SQL vs NoSQL
Database Architecture
Replication
Sharding
DynamoDB
AWS APIGateway and Reverse Proxy
Also, in order to grasp database indexing, you will write a script which will call your API endpoints. This script will be used to benchmark your API. This means that you need to implement a simple benchmarking script. This script will do the following things (please print results to console):
- Insert 10 million tasks and measure the time it takes to insert them.
- List all tasks and measure the time it takes to list them.
- List all done tasks and measure the time it takes to list them.
- List all undone tasks and measure the time it takes to list them.
- Delete all tasks and measure the time it takes to delete them.
Microservices
It is expected to learn these concepts in this challenge:
- Microservice Architecture
- Service Discovery and Registration
- Load Balancing
- Circuit Breaker
- API Gateway
- Event Driven Architecture
- CQRS
- Saga Pattern
- Sync vs Async Communication
- Service Mesh
Virtual Machines
Virtual machines are isolated computing environments that run on a physical machine. The purpose of a virtual machine is running multiple operating systems on a single physical machine. This isolation allows you to run multiple applications on a single physical machine without interfering with each other. EC2 is an example of a virtual machine service.
An hypervisor is a software that runs on a physical machine and manages virtual machines. There are two types of hypervisors: Type 1 and Type 2. Type 1 hypervisors run directly on the hardware and Type 2 hypervisors run on an operating system. KVM is an example of a Type 1 hypervisor and VirtualBox is an example of a Type 2 hypervisor. A host operating system or host machine is an operating system that runs on a physical machine. A guest operating system is an operating system that runs on a virtual machine. A host machine is a physical machine that runs a hypervisor.
Emulation and virtualization are two different concepts used by hypervisors. Emulation is the process of running a software that mimics the behavior of a hardware. This software completely simulates the hardware. As you can imagine, emulation is slow because it has to simulate the hardware. On the other hand, virtualization is the process of running a software / operating system on a virtual machine by running the software directly on the physical hardware. In order to utilize virtualization, the CPU must support virtualization because the isolation is done by the CPU itself. In a virtualized environment, the software again works on a complete isolation without the need of simulating the hardware. If the host machine's CPU architecture is different than the guest machine's CPU architecture, then the hypervisor should use emulation. On the other hand, if virtualization is possible, virtualization should be preferred.
AWS IAM
AWS Roles
AWS SSO
AWS CLI
AWS EC2
EC2 is the short for Elastic Compute Cloud. It is an Aws microservice that provides virtual machines. A cloud hosted virtual machine is useful because you can access it from anywhere/anytime. A connection is needed to access the virtual machine. Opening a connection session is called SSH for Linux systems.
AWS RDS
In order to host a DBMS, you need to host the actual database software in a system. This is where AWS RDS comes into play. AWS RDS is a service that provides DBMS instances. As you expose your DBMS to the internet, you can access it from different sources.
Containers
A computer program is composed of two main parts: code and data. Code is the instructions that are executed by the computer. Data is the information that is processed by the code. In order to run a program, the code must be loaded into the memory. The memory is a temporary storage that is used by the CPU to store the code and data.
Code part of a program is contained in a file which is called an executable file (.exe in Windows, no common extension in Linux and MacOS). An executable file may contain cpu instructions and data that will be loaded into the memory. If the program does not need to load some data into memory before running, then the executable file can run some instructions in order to read the data from a file. Or if the program needs to store a data in a persistent way, then the program can write the data to a file. These operations are done by using operating system's system calls.
Therefore, a large program can be considered as a collection of executable files and data files (sometimes called assets, resources, etc.). However, an executable file may need a library in order to run. A library is a collection of functions that can be used by a program. These files are not executable files, but they are needed by the executable file in order to run (in Windows, they have .dll extension, in Linux they have .so extension, in MacOS they have .dylib extension). In order to run a program, the operating system must load the executable file and the libraries into the memory or the library can be loaded into the memory when the program needs it.
A static binary is an executable file that contains all the code and does not need any library in order to run. On the other hand, a dynamic binary is an executable file that needs a library in order to run. If the library is not found in the system, then the program will not run and will give an error. Also, if a library is updated, then all the programs that use this library also will be affected this change. This may cause a problem because the program may not work as expected.
A container is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings. Therefore every library that is needed by the program is included in the container. This means that the program may run in any environment that shares the same operating system kernel. This is because the container does not contain the operating system itself, but it contains the all other parts that are needed by the program.
This approach solves the problems that are mentioned above. However, it is not enough to bundle all dependencies of a program into a single package. Also we need an isolation mechanism in order to run multiple containers on the same machine in a secure way. This isolation is done by the operating system kernel. The operating system kernel is the core of the operating system that manages the hardware and provides services to the programs. The kernel is responsible for managing the memory, the CPU, the file system, the network, etc. Indeed, each process (a running instance of a program) is isolated from each other in memory. This means that a process cannot access the memory of another process by default. However, filesystem, network, etc. are shared between processes. In order to isolate these parts. Linux introduced namespaces and cgroups. By utilizing these features of the Linux kernel, containers can work on a complete isolation.
Therefore, contaieners may be used for isolation without the need of virtual machines. This comes with a significant performance improvement. For this reason, containers are widely used in the industry. However, in order to work with containers, you need some tools to build container images, run containers, and manage containers. These tools generally comes in a bundled package called container runtime.
Docker
Docker is one of the most popular container runtimes in the industry.
WIP: Container Image WIP: Container Registry WIP: Dockerfile
Docker Compose
App Runner
Serverless
AWS Lambda
CI/CD
DevOps
DevOps is a set of practices that combines software development (Dev) and IT operations (Ops). It aims to shorten the systems development life cycle and provide continuous delivery with high software quality.
Github Actions
Github Secrets and Github Variables
Github Codeowners
PR Environments
Infrastructure as Code
Your app works perferctly. However, when you want to make a simple change, you need to deploy your app manually. If you are a lazy developer, then this challange is for you. In this challange, you will implement a continuous deployment pipeline for your TODO app. This means that, when you push your code to your repository, your app will be automatically deployed to your server. Also, you may notice that, when you made a change you do not have an environment to test your changes. In this challange, you will also separate your environments.
In this challenge, it is expected to learn basics of:
- Continuous Deployment with Github Actions
- Infrastructure as Code with Terraform
- Environment
- AWS SSO
- AWS CLI
Continuous Deployment is a software development practice where code changes are automatically built, tested, and to an environment. This means that, when you push your code to your repository, your app will be automatically deployed to your server.
Environment
An environment is a place where your app is running. Yo
Terraform
Extra Resources
Container Orchestration
Right now, you have a working server side and client side rendered TODO app. Lets imagine that your app has become very popular and you have thousands of users. How you can handle this load? In this challenge, you will learn how to scale your application's computation.
Isolated Environments
A common need in software development is to run your app in an isolated environment. This is especially important in production environments. There are some reasons why you need to run your app in an isolated environment:
- Security: Another app that is running on the same machine can affect your app by using file system, network, or memory. This should be avoided because your app may behave unexpectedly.
- Performance: Also another app that is running on the same machine can affect your app's performance. For instance, another app can consume all the memory or CPU, which can slow down your app.
- Versioning: You may need to run multiple versions of an app on the same machine.
Also, app is not just your backend code. It is also may be a database, a message queue, a cache, a search engine, etc. In order to solve these problems, there are two common approaches: virtual machines and containers.