/ work in progress /.
Introduction.
Each year brings more and more of software and devices connected to the Internet. As more of such are connected, system vulnerabilities increase in numbers as well.
There's deficit in numbers of cybersecurity professionals (in Poland alone, there's need for over 15 000 cybersecurity experts), and it seems that this deficit will increase in numbers as years pass.
Red Team vs. Blue Team.
'Red Team Hackers', 'Ethical Hackers', also called: 'Pentesters' focus on finding vulnerabilities in organization's systems, then report and help Blue Team to 'patch the holes'.
Blue Team are Administrators who are installing, configuring, and monitoring antivirus software, intrusion detection systems, and other protective mechanisms on these devices.
This article concerns more about theory and is mostly of interrest for future Blue Team members. It touches only basics, however. Blue Team aspirants should deepen the knowledge using other means as well.
Red Team members are meant to cooperate & communicate with Blue Team, hence why this article should be useful for them as well.
CIA Triad.
C - Confidentiality
- Ensuring that informations are available only for authorized personnel
I - Integrity
- Protecting data from unauthorized modification or deletion
A - Availability
- Ensuring that informations are available for authorized personnel
Methods for Ensuring Confidentiality:
- Security Policy (Categorizing Data as either Confidential or Public,
and ensuring that only authorized personnel has access to Confidential Data)
- Encryption
- Access Control (Physical and Multifactored Access Control)
Securing a device often comes at the cost of convenience of use. Protected laptop computer, for example, should be still useful and use-conveniet for authorized personnel. So, in practice, security means should not be too excessive. We should not request, for example, 128-characters-long random passwords from users.
Meaning of Integrity:
- Data should be protected from unauthorized modification
- Example attacks on Data Integrity:
- Modifying company's payments lists
- Modifying company's webpage conent files
- Might lead to disastrous consequences for a company
Meaning of Availability & Methods for Ensuring Availability:
- Ensuring continuity of work of systems and applications
- Redundancy (extra resources for case of malfunctions)
- Keeping backups (for swift restoration of data and systems in case of malfunction)
Examples & Solutions for Availability Violations:
- DDoS attacks (Resources Overload might cause troubles in accessibility)
- Viral Marketing (Sudden increase in Web Traffic might lead to overloading of
server resources)
- Solutions:
- Scalability
- Load Balancing
- Firewalls
- Monitoring
Additional characteristics of CIA model:
- Authentication & Authorization (verifying user's identity and privileges)
- Non-repudiation (provides proof of the origin, authenticity and integrity of data.
It provides assurance to the sender that its message was delivered, as well as proof
of the sender's identity to the recipient)
This way, neither party can deny that a message was sent, received and processed)
- Robustness (ability of a computer system to cope with errors during execution
and cope with erroneous input)
- Compliance (fulfilling legal and regulatory requirements)
- Privacy (protecting user privacy and personal data)
Cybersecurity as Process - CSF2.0
Overview:
- Cybersecurity should be treated as continuous process, not as single, one-time,
100% complete solution
- Every day new attacks & hacking techniques appear. Hacking tools are under constant
development, with time more advanced and more sophisticated hacking tools appear
- There's need for constant updating of security strategies
- Organizations keep deploying new technological solutions. New technologies might
introduce new vulnerabilities, new security holes
Being ready for incidents:
- Preparation (developing incident reaction plans)
- Detection (swift identification of potential threats)
- Reaction (immediate reaction to appearing incidents)
Process approach to cybersecurity can ensure readiness for action at any moment.
Process approach to cybersecurity can make compliance with legal and industry requirements easier.
Process approach in practice:
- Planning (developing strategies and objectives of cybersecurity)
- Deployment (implementing planned actions & controls)
- Assesment (efficiency analysis of deployed solutions)
- Perfection (constant refinement of security practices)
- Continuity & cyclicity (keep repeating above solutions in cycles)
Elasticity & Adaptation:
- Fast adaptation to appearing threats
- Adaptation to changing technologies & IT architecture
- Adaptation to changing business requirments, to changing organization needs
Integration with business processes (cybersecurity should not be treated as a
separate concern, should a part of business processes):
- Risk Management (integrating cybersecurity with overall risk management)
- Products Development (take cybersecurity into consideration with products'
lifecycles)
- Customer Service (integrating cybersecurity with customer support practices)
Cyber Security Framework 2.0 (CSF2.0):
- Created & Developed by NIST (National Institute of Standards & Technology in USA)
- Objective: helps with Risk Management in Cybersecurity
- Universality: can be tailored for small or large companies & organizations
CSF Elasticity:
- Elastic: CSF is an elastic tool, not stiff regulation or standard
- Adjustment: can be tailored for specific organization needs
- Pointers: generic approach, with possibility of custom implementation
CSF2.0 Components:
- Core (center of framework, with key functionalities)
- Profile (description of current and target cybersecurity approach)
- Tiers (characteristics of rigors of actions related with risk management)
CSF2.0 Core:
Govern:
- Describes strategy of Risk Management in organization
- 'Surrounds' and manages five other CSF2.0 components
- Includes determining roles & responsibilites, who is responsible for what
- Manages supply chain(s)
- Sets policies related with cybersecurity
Identify (understanding of what we want to protect):
- Understanding (increasing understanding of current risks in cybersecurity)
- Resources Management (identification & management of organization's resources)
- Risk Assesment (analysis & evaluation of potential threats)
Protect:
- Security Tools (deploying proper security tools for risk management)
- Access Control (managing authentication & authorization, and raising
users' cybersecurity awareness)
- Data Security (protecting organization's sensitive informations)
Detect:
- Monitoring (constant monitoring of systems and networks)
- Analysis (analysis of incidents & anomalies in realtime)
- Alerts (generating alerts of potential security incidents)
Respond:
- Reaction Plan (preparation and deployment of incidents reaction plans)
- Incident Analysis (detailed analysis of detected cybersecurity incident)
- Softening Effects (actions meant for minimizing effects of security incident)
- also: Reporting & Communication
Recover:
- Recovery Plan (executing plan meant for restoring normal functioning)
- Communication (informing interrested parties about recovery processes)
- Post-Incident Analysis (drawing conclusions & refining processes)
CSF2.0 Profiles:
- Current Profile (description of current organization's cybersecurity state)
- Target Profile (description of desired cybersecurity state to achieve)
CSF2.0 Tiers:
Tiers determine how organization approaches and manages risks on the whole, how much of resources are spent for which risks.
Organization should consider how dangerous which risks are, resources available, and possibility of certain solution deployments.
Choice of Tiers for risks depends on organization. Not always higher tiers mean better security. Organization should choose which tiers to assign for what risks, depending on organization's resources and current needs.
Integrating CSF2.0 with overall risk management:
- Balance (treating cybersecurity risks as equal with other risks)
- Integration (including cybersecurity in overall risk management)
- Development (constant development of risk management processes)
Vulnerabilities and Metrics - CVSS.
/ to be done /.
Threat and Threats Modelling.
/ to be done /.
Risk Management.
/ to be done /.
Risk Metrics.
/ to be done /.
Attack & Incident Reaction Plans.
/ to be done /.
Who is Attacker & Attack Types.
/ to be done /.
Project Wraithstar.
... this blog's author is still learning, so much to consider & experiment with.
... i hope that as i grow in understanding, ideas that i explore here will turn into the science with great depth - that's one of my goals in this blog.
Thursday, 19 December 2024
Sunday, 15 December 2024
Kali Linux Reference for Pentesters.
Pentesters.
Pentesters are ethical hackers who look for vulnerabilities in target system, and report results to company that hired them, and help to patch (fix) found vulnerabilities before criminals use these vulnerabilities earlier.
Pentesters are also called: 'Red Team Hackers'.
Operating Systems for Pentesters.
Beside Kali Linux (successor of the BackTrack Linux), there's Parrot OS, BlackArch Linux, BackBox, probably more as well. All of these are Operating Systems made for Pentesters.
But Kali Linux is industry's standard, widely used OS for Pentesters.
Directory Structure in Kali Linux:
Commands & Important ideas.
Basic Commands:
- pwd: print working directory
- cd: change directory
- sudo: executing commands with root privileges
- ls: listing directory contents
- ls -l: as ls, but more details provided
- ls -a: as ls, but listing all files, not ignoring hidden ones
- man: displaying manual for commands
- man -f: short explaination of a command
- apropos: searching for commands using keywords
- history: displaying used commands history
Search Commands:
- find: detailed search
- locate: fast search using informations stored in database
- which : searching for executable files locations
- whereis: searching for executable files and related files locations
also, updating database for use of 'locate' command can be done using: 'sudo updatedb' command.
File & directory operations:
- touch: creating empty files
- mkdir: creating directories
- rm: removing files and/or directories
- tree: displaying directory tree structure
- cp: copying files and/or directories
- mv: moving files and/or directories
- echo: printing text
- cat: displaying file's contents
- wc -l: counting lines
- sort: sorting content
Streams:
- stdin: standard input (0)
- stdout: standard output (1)
- stderr: standard error (2)
Redirect operators:
- redirecting stdout (overwriting file): 1>
- redirecting stdout (adding to file): 1>>
- passing stdout to another (next) command (pipe): |
- redirecting stdin: <
- redirecting stderr (overwriting file): 2>
- redirecting stderr (adding to file): 2>>
Collecting System Info:
Informations about users:
- whoami: displays current user's name
- id: detailed information about current user & groups
- who: list of logged in users
Informations about system:
- hostname: name of the host
- uname (-a): (detailed) information about operating system
- env: displays all environment variables
- ps (aux): displays list of processes of all users
- top: monitoring processes in realtime
Network informations:
- ifconfig: displays network interfaces & their configurations
- ip addr: displays ip addresses
- netstat: displays network connections
- ss: socket statistics
Informations about devices:
- df -h: disk space usage, printed in 'human-readable' form
- lsblk: block devices list
- lsusb: USB devices list
- lspci: PCI devices list
Linux Text Editors:
Nano:
nano [file_name]: open/create file
Ctrl + G: help
Ctrl + X: exit
Ctrl + /n: move to line n
Ctrl + K: delete line
Ctrl + F: searching
Vim:
vim [file_name]: open file
i: insert text mode
ESC: return to command mode
dd: delete line
yy: copy line
p: paste
/text: search for text
:w: save
:q: exit
:wq or :x: save & exit
:q!: exit without saving
:set number: display lines numbers
vimtutor: interactive tutorial
Privileges:
Basic Privileges:
- r: read
- w: write
- x: execute
Categories:
- user
- group
- others
Main commands:
- chmod: change privileges
- chmod u+w: adding write privileges for file's owner (user)
- chmod g-w: remove write privileges for file's owner's group
- chmod o+x: adding execute privileges for other users
- chmod 644: privileges in octal notation
- chmod 755: standard privileges for directories
Special ('sticky') bits:
- SUID: execute with owner's privileges
- SGID: execute with group's privileges
System files:
- /etc/passwd: informations about users
- /etc/shadow: encrypted passwords
- /etc/group: informations about groups
Users management:
Managing users:
- useradd: creating user
- useradd -m: creating with home directory
- useradd -s: defining default shell
- useradd -g: setting primary group
- usermod: modifying user
- usermod -g: changing primary group
- usermod -aG: adding additional groups
- deluser: deleting user
- deluser --remove-home: removing with home directory
Managing groups:
- addgroup: creating group
- groups: displaying user's groups
- deluser user group: removing user from group
Managing sudo:
- sudo -l: checking sudo privileges
- visudo: safe editing of sudoers
- su: switching user
- passwd: changing password
Configuration files:
- /etc/sudoers: sudo configuration
- /etc/sudoers.d/: additional configuration files
Installing software:
APT (Advanced Package Tool):
- sudo apt update: refreshing packets list
- sudo apt upgrade: upgrades all packages to newest versions
- apt list --upgradable: shows list of available package upgrades
- apt list --installed: shows list of installed packages
- apt search [name]: searching for package
- apt show [name]: details about package
- sudo apt install [name]: installing package
- sudo apt remove [nazwa]: remove package
DPKG:
- dpkg -l: list of installed dpkg packages
- dpkg -l | grep [name]: search within packages
Additional tools:
- git clone [url] - cloning git repository
- pip install [name] - installing python packages
Useful options:
- --only-upgrade: upgrading only specified package
- -y: automatic install confirmation
Processes & services:
Managing processes:
- ps aux: processes list
- ps auxf: processes list displayed in form of tree
- kill [PID]: termination of a process
- kill -9 [PID]: forced termination of a process
- killall [name]: terminating all processes with given name
- jobs: listing background tasks
- fg %[numer]: moving task to foreground
- ctrl + Z: sending process into background
- &: executing process in background
Managing services:
- systemctl list-units --type=service --all: shows services list
- service --status-all: shows status of all services
- systemctl status [service]: detailed status
- systemctl start/stop/restart [service]: service management
- service [service] start/stop/restart: alternative service management
- journalctl -u [service]: service's logs
Monitoring:
- top - monitoring processes in realtime
- ps auxf | grep [name] - searching for processes
Wordlists.
Wordlists can be used, for example:
- for executing dictionary attacks that find passwords,
- for finding file names when we can't just list the containing directory's contents
(webpage server's directories sometimes deny listing files in these directories).
One of the most popular wordlist is named 'rockyou.txt'.
In Kali Linux it's available in compressed form, in following path:
/usr/share/wordlists/rockyou.txt.gz
Pentesters are ethical hackers who look for vulnerabilities in target system, and report results to company that hired them, and help to patch (fix) found vulnerabilities before criminals use these vulnerabilities earlier.
Pentesters are also called: 'Red Team Hackers'.
Operating Systems for Pentesters.
Beside Kali Linux (successor of the BackTrack Linux), there's Parrot OS, BlackArch Linux, BackBox, probably more as well. All of these are Operating Systems made for Pentesters.
But Kali Linux is industry's standard, widely used OS for Pentesters.
Directory Structure in Kali Linux:
/bin (binaries) | This directory contains Linux binaries like the cd and ls commands. |
/sbin (system binaries) | This directory holds system binary files that serve as administrative commands (like fdisk). |
/boot | This directory contains the Linux bootloader files. |
/dev (devices) | This directory contains the device configuration files (like /dev/null ). |
/sys | This is similar to /dev, which contains configurations about devices and drivers. |
/etc (etcetera) | This directory contains all the administration system files (like /etc/passwd shows all the system users in Kali Linux). |
/lib (libraries) | This directory hods the shared libraries for the binaries inside /bin and /sbin. |
/proc (processes) | This directory contains the processes and kernel information files. |
/lost+found | As in the name, this directory contains the files that have been recovered. |
/mnt (mount) | This directory contains the mounted directories (example, a remote file share). |
/media | This directory holds the removable media mounted directories (like DVD). |
/opt (option) | This directory is used for add‐on software package installation. It is also used when installing software by users (example, hacking tools that you download from GitHub). |
/tmp (temporary) | This is a temporary folder used temporarily, the holdings are wiped after each reboot. The tmp folder is a good place to download our tools for privilege escalation once we got a limited shell. |
/usr (user) | This directory contains many sub-directories. In fact, /usr/share/ is a folder that we need to memorize because most of the tools that we use in Kali Linux (like Nmap, Metasploit, etc.) are stored there, and it also contains the wordlist dictionary files (/usr/share/wordlists). |
/home | This is the home for Kali Linux users (example /home/kali/). |
/root | Home directory for root user. |
/srv (serve) | This folder contains some data related to system server functionalities (like data for FTP servers). |
/var (variable) | This folder contains variable data for databases, logs, and websites. For an example, /var/www/html/ contains the files for the Apache2 web server. |
/run (runtime) | This directory holds runtime system data (like currently logged‐in users). |
Commands & Important ideas.
Basic Commands:
- pwd: print working directory
- cd: change directory
- sudo: executing commands with root privileges
- ls: listing directory contents
- ls -l: as ls, but more details provided
- ls -a: as ls, but listing all files, not ignoring hidden ones
- man: displaying manual for commands
- man -f: short explaination of a command
- apropos: searching for commands using keywords
- history: displaying used commands history
Search Commands:
- find: detailed search
- locate: fast search using informations stored in database
- which : searching for executable files locations
- whereis: searching for executable files and related files locations
also, updating database for use of 'locate' command can be done using: 'sudo updatedb' command.
File & directory operations:
- touch: creating empty files
- mkdir: creating directories
- rm: removing files and/or directories
- tree: displaying directory tree structure
- cp: copying files and/or directories
- mv: moving files and/or directories
- echo: printing text
- cat: displaying file's contents
- wc -l: counting lines
- sort: sorting content
Streams:
- stdin: standard input (0)
- stdout: standard output (1)
- stderr: standard error (2)
Redirect operators:
- redirecting stdout (overwriting file): 1>
- redirecting stdout (adding to file): 1>>
- passing stdout to another (next) command (pipe): |
- redirecting stdin: <
- redirecting stderr (overwriting file): 2>
- redirecting stderr (adding to file): 2>>
Collecting System Info:
Informations about users:
- whoami: displays current user's name
- id: detailed information about current user & groups
- who: list of logged in users
Informations about system:
- hostname: name of the host
- uname (-a): (detailed) information about operating system
- env: displays all environment variables
- ps (aux): displays list of processes of all users
- top: monitoring processes in realtime
Network informations:
- ifconfig: displays network interfaces & their configurations
- ip addr: displays ip addresses
- netstat: displays network connections
- ss: socket statistics
Informations about devices:
- df -h: disk space usage, printed in 'human-readable' form
- lsblk: block devices list
- lsusb: USB devices list
- lspci: PCI devices list
Linux Text Editors:
Nano:
nano [file_name]: open/create file
Ctrl + G: help
Ctrl + X: exit
Ctrl + /n: move to line n
Ctrl + K: delete line
Ctrl + F: searching
Vim:
vim [file_name]: open file
i: insert text mode
ESC: return to command mode
dd: delete line
yy: copy line
p: paste
/text: search for text
:w: save
:q: exit
:wq or :x: save & exit
:q!: exit without saving
:set number: display lines numbers
vimtutor: interactive tutorial
Privileges:
Basic Privileges:
- r: read
- w: write
- x: execute
Categories:
- user
- group
- others
Main commands:
- chmod: change privileges
- chmod u+w: adding write privileges for file's owner (user)
- chmod g-w: remove write privileges for file's owner's group
- chmod o+x: adding execute privileges for other users
- chmod 644: privileges in octal notation
- chmod 755: standard privileges for directories
Special ('sticky') bits:
- SUID: execute with owner's privileges
- SGID: execute with group's privileges
System files:
- /etc/passwd: informations about users
- /etc/shadow: encrypted passwords
- /etc/group: informations about groups
Users management:
Managing users:
- useradd: creating user
- useradd -m: creating with home directory
- useradd -s: defining default shell
- useradd -g: setting primary group
- usermod: modifying user
- usermod -g: changing primary group
- usermod -aG: adding additional groups
- deluser: deleting user
- deluser --remove-home: removing with home directory
Managing groups:
- addgroup: creating group
- groups: displaying user's groups
- deluser user group: removing user from group
Managing sudo:
- sudo -l: checking sudo privileges
- visudo: safe editing of sudoers
- su: switching user
- passwd: changing password
Configuration files:
- /etc/sudoers: sudo configuration
- /etc/sudoers.d/: additional configuration files
Installing software:
APT (Advanced Package Tool):
- sudo apt update: refreshing packets list
- sudo apt upgrade: upgrades all packages to newest versions
- apt list --upgradable: shows list of available package upgrades
- apt list --installed: shows list of installed packages
- apt search [name]: searching for package
- apt show [name]: details about package
- sudo apt install [name]: installing package
- sudo apt remove [nazwa]: remove package
DPKG:
- dpkg -l: list of installed dpkg packages
- dpkg -l | grep [name]: search within packages
Additional tools:
- git clone [url] - cloning git repository
- pip install [name] - installing python packages
Useful options:
- --only-upgrade: upgrading only specified package
- -y: automatic install confirmation
Processes & services:
Managing processes:
- ps aux: processes list
- ps auxf: processes list displayed in form of tree
- kill [PID]: termination of a process
- kill -9 [PID]: forced termination of a process
- killall [name]: terminating all processes with given name
- jobs: listing background tasks
- fg %[numer]: moving task to foreground
- ctrl + Z: sending process into background
- &: executing process in background
Managing services:
- systemctl list-units --type=service --all: shows services list
- service --status-all: shows status of all services
- systemctl status [service]: detailed status
- systemctl start/stop/restart [service]: service management
- service [service] start/stop/restart: alternative service management
- journalctl -u [service]: service's logs
Monitoring:
- top - monitoring processes in realtime
- ps auxf | grep [name] - searching for processes
Wordlists.
Wordlists can be used, for example:
- for executing dictionary attacks that find passwords,
- for finding file names when we can't just list the containing directory's contents
(webpage server's directories sometimes deny listing files in these directories).
One of the most popular wordlist is named 'rockyou.txt'.
In Kali Linux it's available in compressed form, in following path:
Sunday, 1 September 2024
Computers, Memory Pyramid & Code Size Optimization.
What is a Computer?
In Computer Sciences, Computer - by definition - is processor with memory and input/output devices. Any electronic device that has these is considered Computer. This includes Smartphones and many other tools.
Memory, Persistent or Transient.
There are two types of memory / pl: 'Są dwa rodzaje pamięci' /:
- Persistent / pl: 'Trwała' /,
- Transient / pl: 'Ulotna' /.
Persistent objects are those which continue to exist even after the program that created them has stopped running.
Transient objects cease to exist when program that created them stops.
Pyramid of Needs.
There are many types of memory, differing in price and speed of access.
Starting from the most expensive but fastest, there are:
- processor's registers,
- layers of the processor's cache (L1-L3, for example),
- RAM (Random Access Memory),
- persistent SSD/HDD storage.
Code Size optimization.
Smaller programs can be very quick in their execution.
When the whole program fits - for example - in L2 Processor's Cache, there's no need to reach RAM via BUS, so the code runs very quickly - as it's closer to the processor than RAM.
What if a Program doesn't fit in Transient Memory?
When a program needs to be executed, it needs to be loaded into the transient memory first.
However, Modern Operating Systems can send currently unused Program's parts & other Resources / for example: graphics image files, sound files and/or text files / from Transient Memory to Persistent SDD/HDD Memory and retrieve other Resources/Part(s) from Persistent Memory to Transient Memory / Usually from disk to RAM /.
/ pl: 'Współczesne Systemy Operacyjne mogą wysłać aktualnie niewykorzystywane części Oprogramowania i innych zasobów na dysk... i sprowadzić inne zasoby/części z pamięci trwałej do ulotnej, najczęściej do pamięci RAM' /.
Let's note, however, that loading/storing data in persistent memory is much slower than loading/storing from/to Transient RAM.
This is an automated operation in Modern Operating Systems, so programmers do not need to worry so much about that. Computers just slow down sometimes - and SSD/HDD becomes quite busy, when doing that.
This sometimes causes 'Flickering' / pl: 'Migotanie', 'Szamotanie' /, however. Code & Data is loaded/unloaded from/to persistent memory too slowly and can cause a Computer System to slow down or crash, as the Computing Resources run out. / Mostly CPU usage & Memory usage /.
So - in theory at least - a Computer can try to run larger programs than Computer has RAM.
Often it fails, but in theory this can work well.
In Computer Sciences, Computer - by definition - is processor with memory and input/output devices. Any electronic device that has these is considered Computer. This includes Smartphones and many other tools.
Memory, Persistent or Transient.
There are two types of memory / pl: 'Są dwa rodzaje pamięci' /:
- Persistent / pl: 'Trwała' /,
- Transient / pl: 'Ulotna' /.
Persistent objects are those which continue to exist even after the program that created them has stopped running.
Transient objects cease to exist when program that created them stops.
Pyramid of Needs.
There are many types of memory, differing in price and speed of access.
Starting from the most expensive but fastest, there are:
- processor's registers,
- layers of the processor's cache (L1-L3, for example),
- RAM (Random Access Memory),
- persistent SSD/HDD storage.
Code Size optimization.
Smaller programs can be very quick in their execution.
When the whole program fits - for example - in L2 Processor's Cache, there's no need to reach RAM via BUS, so the code runs very quickly - as it's closer to the processor than RAM.
What if a Program doesn't fit in Transient Memory?
When a program needs to be executed, it needs to be loaded into the transient memory first.
However, Modern Operating Systems can send currently unused Program's parts & other Resources / for example: graphics image files, sound files and/or text files / from Transient Memory to Persistent SDD/HDD Memory and retrieve other Resources/Part(s) from Persistent Memory to Transient Memory / Usually from disk to RAM /.
/ pl: 'Współczesne Systemy Operacyjne mogą wysłać aktualnie niewykorzystywane części Oprogramowania i innych zasobów na dysk... i sprowadzić inne zasoby/części z pamięci trwałej do ulotnej, najczęściej do pamięci RAM' /.
Let's note, however, that loading/storing data in persistent memory is much slower than loading/storing from/to Transient RAM.
This is an automated operation in Modern Operating Systems, so programmers do not need to worry so much about that. Computers just slow down sometimes - and SSD/HDD becomes quite busy, when doing that.
This sometimes causes 'Flickering' / pl: 'Migotanie', 'Szamotanie' /, however. Code & Data is loaded/unloaded from/to persistent memory too slowly and can cause a Computer System to slow down or crash, as the Computing Resources run out. / Mostly CPU usage & Memory usage /.
So - in theory at least - a Computer can try to run larger programs than Computer has RAM.
Often it fails, but in theory this can work well.
Monday, 29 April 2024
Logic & Axioms.
There are many Logics, based on different Axioms.
/ EN: 'Axiom' = PL: 'Aksjomat' /.
Axioms are statements that are not proven, but assumed as true, taken on faith.
/ EN: 'assumption' = PL: 'założenie' /.
Depending on the Axioms used, Theorems can be proven or disproven, and the whole Mathematical & Logical Apparatus can be developed.
Basing on Boolean Algebra, double negation evaluates to confirmation, but in some languages - polish for example - double negation does not mean confirmation, it does mean emphasis on negation, giving negation more power.
There are rules for negating Quantifiers as well.
/ EN: 'negation' = PL: 'zaprzeczenie' /,
/ EN: 'confirmation' = PL: 'potwierdzenie' /,
/ EN: 'emphasis' = PL: 'nacisk' /,
/ EN: 'quantifier' = PL: 'kwantyfikator' /.
/ według Algebry Boole'a i Teorii Kwantyfikatorów wylicza się do:
'istnieje taki moment w czasie kiedy zgodzę się na te warunki'. /
/ a w mowie potocznej znaczy: 'nie istnieje taki moment w czasie kiedy zgodzę się na te warunki'. /
Therefore, Speech of the Art, Literature, can have the Logic based on different Axioms than Boolean Algebra.
We can say that: 'Life is more than Boolean Algebra and Quantifiers' when we want to use casual, non-logical talks.
When Boolean Algebra and Quantifiers are useful then? It's useful in computer programming, or when we want to talk logically and precisely, or when we want to express our wishes logically and precisely. But when we opt for logical and precise speech, let's make sure first that other people we talk with understand our logic.
We can also define the addition operation / it's an Axiom too / differently as well.
We can have an exception:
1+1 = 2, 1+2 = 3, 2+1 = 3, 1+3 = 4, 3+1 = 4, 2+2 = 5, 1+4 = 5, 4+1 = 5, 2+3 = 5, 3+2 = 5, 1+5 = 6, 5+1 = 6, 2+4 = 6, ...
We can also redefine the addition operation differently on the more general, more universal scale:
n+2 in our redefined addition operation is n+2+1 in classical addition operation.
By doing so, by changing Axioms, we just have revolutionized the Mathematics. ;)
Many different theorems apply now, but at least we know that we can make expression 2+2 = 5 to be evaluated as true in a certain Context - even if this brings more or less desired effects in process ;).
We are free to assume any Axioms we want, examples can be multiplied infinitely.
Which Logic is 'better' than other, then?
... it depends on the assumed Criteria, which might be Axioms as well.
Tuesday, 6 February 2024
The Enigma Cipher of WW2 & the Turing Machine.
Polish and British mathematicians were among the best of people who cracked the Adolph Hitler's cipher named Enigma, it happened during the World War 2nd.
Enigma breaking was hard, and the mathematicians were hunted by Germany's spies.
Enigma evolved, so parts of the cipher were to be cracked again and again. It was not about making an automaton once, and letting it work for the rest of WW2 ... but statistically it worked so the effort was continued.
Doing maths when time flew and lives were at stake.... so stressful. The Germany's spies added to the dangers & to the stress too.
Polish mathematicians had one of few of first computers ... it was nicknamed: 'Bomb', for it was so big invention. It increased efficiency of the enigma cipher cracking.
And there was Alan Turing's effort of course. He was a British scientist who laid foundation-theory of computer's construction.
His thinking is still present in computer sciences of modern days.
(We were taught about Languages & Automatons in Warsaw University when i was studying computer sciences. Turing's Machine was a part of this lecture).
> [ https://en.wikipedia.org/wiki/Turing_machine ].
The Turing Machine is programmed in a similar way to programming the Register Machine.
> [ https://en.wikipedia.org/wiki/Register_machine ].
--
Sources:
1. 'Cubits & Shrodinger's Cat. From Turing Machine to Quantum Computers' by John Gribbin.
(Polish Edition).
2. The internet (wikipedia & the ważniak mostly) and my own thinking.
3. My (unfinished because of health problems) education at Warsaw's University (Mathematics, Informatics & Mechanics Faculty).
Enigma breaking was hard, and the mathematicians were hunted by Germany's spies.
Enigma evolved, so parts of the cipher were to be cracked again and again. It was not about making an automaton once, and letting it work for the rest of WW2 ... but statistically it worked so the effort was continued.
Doing maths when time flew and lives were at stake.... so stressful. The Germany's spies added to the dangers & to the stress too.
Polish mathematicians had one of few of first computers ... it was nicknamed: 'Bomb', for it was so big invention. It increased efficiency of the enigma cipher cracking.
And there was Alan Turing's effort of course. He was a British scientist who laid foundation-theory of computer's construction.
His thinking is still present in computer sciences of modern days.
(We were taught about Languages & Automatons in Warsaw University when i was studying computer sciences. Turing's Machine was a part of this lecture).
> [ https://en.wikipedia.org/wiki/Turing_machine ].
The Turing Machine is programmed in a similar way to programming the Register Machine.
> [ https://en.wikipedia.org/wiki/Register_machine ].
--
Sources:
1. 'Cubits & Shrodinger's Cat. From Turing Machine to Quantum Computers' by John Gribbin.
(Polish Edition).
2. The internet (wikipedia & the ważniak mostly) and my own thinking.
3. My (unfinished because of health problems) education at Warsaw's University (Mathematics, Informatics & Mechanics Faculty).
Friday, 24 March 2023
Realtime Systems.
Introduction.
Real-Time systems are computer systems in which succesful task performance depends on two factors:
- succesful result of computation,
- exact deadline time in which task was performed.
When a task execution time exceeds it's time deadline, we can say that system failed.
Hard, Firm & Soft Real-time Systems.
In the 'Hard Real-time Systems' missing a deadline is a total system failure. Hard realtime systems are created when missing a deadline can result in hardware damage or costs lives or health.
In 'Firm Real-time Systems' infrequent deadline misses are tolerable, but may degrade the system's quality of service. The usefulness of a result is zero after its deadline, but no damage or personnel loss occurs.
In 'Soft Real-time Systems' the usefulness of a result degrades after its deadline, thereby degrading the system's quality of service.
Requirements.
For a computer system to meet criteria of the 'Hard Real-time System', requirements are:
- well understood & fast enough hardware,
- real-time operating system (for example: RTLinux),
- every software piece must adhere to the 'hard real-time requirements'; software must generate results in the deadline time.
Real-Time systems are computer systems in which succesful task performance depends on two factors:
- succesful result of computation,
- exact deadline time in which task was performed.
When a task execution time exceeds it's time deadline, we can say that system failed.
Hard, Firm & Soft Real-time Systems.
In the 'Hard Real-time Systems' missing a deadline is a total system failure. Hard realtime systems are created when missing a deadline can result in hardware damage or costs lives or health.
In 'Firm Real-time Systems' infrequent deadline misses are tolerable, but may degrade the system's quality of service. The usefulness of a result is zero after its deadline, but no damage or personnel loss occurs.
In 'Soft Real-time Systems' the usefulness of a result degrades after its deadline, thereby degrading the system's quality of service.
Requirements.
For a computer system to meet criteria of the 'Hard Real-time System', requirements are:
- well understood & fast enough hardware,
- real-time operating system (for example: RTLinux),
- every software piece must adhere to the 'hard real-time requirements'; software must generate results in the deadline time.
Classic Concurrency Problems.
1. Mutual Exclusion.
Let's assume that two processess want to access critical section of code. They are acting as such:
process P;
begin
while true do
begin
personal_affairs;
begining_protocol;
critical_section;
ending_protocol;
end
end;
Critical section is this program fragment, that can be executed by at most one process at once.
We assume that each process which enters critical section, will leave it in finite time.
See also: Petersen's Algorithm.
2. Producers & Consumers.
There are P>0 processes in system, which produce certain data, and K>0 processes which receive data from producers. Between producers and consumers there can be buffer with capacity B, whose task is balancing temporary differences during processes execution time. Processes that produce data we'll call producers, and processes receiving data --- consumers. Task is about synchronizing work of producers and consumers, such as that:
* Consumer would wait for data receive in situation when buffer is empty.
* Producer, when putting data in buffer would not overwrite data already written, but not received yet by consumer. It requires stopping temporarily producer in situation when there is no empty space in buffer.
* If many consumers wait for when data arrives in buffer, and if constantly new data is produced, then each waiting consumer will get something from buffer.
* Won't happen such situation, that certain consumer will wait infinitely for getting data, if data arrives in buffer constantly.
* If many producers wait for buffer's free space, and consumers constantly get something from buffer, then each of waiting producers will be able to put something into buffer. There won't happen situation such as certain producer will wait infinitely, if constantly something is taken from buffer.
There are variants to this problem:
* Buffer might be infinite.
* Cyclic buffer with finite space.
* No buffer at all.
* Many producers or only one.
* Many consumers or only one.
* Data might be produced and consumed at quicker rate (more than one unit at once).
* Data has to be read in writing order, or not.
Producers and consumers problem is abstraction of many situations existing in computer systems, for example: keyboard data write to buffer by keyboard device driver, and it's read by operating system.
3. Readers and Writers.
There are C>0 processes working in system. They read certain data. There are P>0 processes which write data. Processes writing data we'll call writers, processes reading data --- readers. Moment when processes have access to data, we'll call 'reading room visit' or 'reading room stay'.
Let's notice that many processes can read data at once. If someone wants to modify that data, then it's reasonable to block access to this data for all other processes for the time of read. It prevents read of inconsistent data (for example, partially modified data). Overall schema for processes work is such:
process Reader;
begin
repeat
personal_affairs;
begining_reader_protocol;
READ;
ending_reader_protocol;
until false
end
process Writer;
begin
repeat
personal_affairs;
begining_writer_protocol;
WRITE;
ending_writer_protocol;
until false
end
Beginning and ending protocols of selected processes should be written in a way, that allows for these conditions to be met:
* Many readers should have simultaneous access to reading room.
* If in reading room there's writer, then no one else does not write or read.
* Each reader, which wishes for data read, at some point will read them.
* Each writer, which wishes for data modification, at some point will write such modifications.
There are variants to this problem:
* In reading room many readers can spend their time at once.
* Reading room may have limited capacity.
* Writers may have priority over readers (but then we'll resign from readers liveness).
* Readers may have priority over writers (but then we'll resign from writers liveness).
4. Five Philosophers.
This problem does not have practical analogies, unlike previous classic problems, but it very well illustrates problems happening when concurrent programs are made.
Five philosophers dine by round table. Before each there's plate. Between plates lie forks. In middle of the table there's serving dish with fish. Each philosopher thinks. When he gets hungry, he reaches for forks laying at his right and left side, then starts eating. When he's done eating, he puts forks away and again devotes himself to thinking.
Scheme of such philospoher is such then:
process Philospher (i: 0..4);
begin
repeat
think;
beginning_protocol;
eating;
ending_protocol;
until false
end;
Task is to write beginning and ending protocols, such as following conditions would be met:
* Only one philospoher ate with the same fork at the same time.
* Each of the philosophers ate only and always with two (and always those which lay near his plate) forks.
* No philosopher would die of starvation.
* Also we want that each of the philosophers would act the same way.
Solution to this riddle is to use waiter to let only four philosophers at the dining table at the same time, and let fifth wait. When philosopher is done eating, he leaves table and joins queue and can only return to table when waiter allows him to do this.
See also, if You wish: Basics of Concurrent Programming.
Let's assume that two processess want to access critical section of code. They are acting as such:
process P;
begin
while true do
begin
personal_affairs;
begining_protocol;
critical_section;
ending_protocol;
end
end;
Critical section is this program fragment, that can be executed by at most one process at once.
We assume that each process which enters critical section, will leave it in finite time.
See also: Petersen's Algorithm.
2. Producers & Consumers.
There are P>0 processes in system, which produce certain data, and K>0 processes which receive data from producers. Between producers and consumers there can be buffer with capacity B, whose task is balancing temporary differences during processes execution time. Processes that produce data we'll call producers, and processes receiving data --- consumers. Task is about synchronizing work of producers and consumers, such as that:
* Consumer would wait for data receive in situation when buffer is empty.
* Producer, when putting data in buffer would not overwrite data already written, but not received yet by consumer. It requires stopping temporarily producer in situation when there is no empty space in buffer.
* If many consumers wait for when data arrives in buffer, and if constantly new data is produced, then each waiting consumer will get something from buffer.
* Won't happen such situation, that certain consumer will wait infinitely for getting data, if data arrives in buffer constantly.
* If many producers wait for buffer's free space, and consumers constantly get something from buffer, then each of waiting producers will be able to put something into buffer. There won't happen situation such as certain producer will wait infinitely, if constantly something is taken from buffer.
There are variants to this problem:
* Buffer might be infinite.
* Cyclic buffer with finite space.
* No buffer at all.
* Many producers or only one.
* Many consumers or only one.
* Data might be produced and consumed at quicker rate (more than one unit at once).
* Data has to be read in writing order, or not.
Producers and consumers problem is abstraction of many situations existing in computer systems, for example: keyboard data write to buffer by keyboard device driver, and it's read by operating system.
3. Readers and Writers.
There are C>0 processes working in system. They read certain data. There are P>0 processes which write data. Processes writing data we'll call writers, processes reading data --- readers. Moment when processes have access to data, we'll call 'reading room visit' or 'reading room stay'.
Let's notice that many processes can read data at once. If someone wants to modify that data, then it's reasonable to block access to this data for all other processes for the time of read. It prevents read of inconsistent data (for example, partially modified data). Overall schema for processes work is such:
process Reader;
begin
repeat
personal_affairs;
begining_reader_protocol;
READ;
ending_reader_protocol;
until false
end
process Writer;
begin
repeat
personal_affairs;
begining_writer_protocol;
WRITE;
ending_writer_protocol;
until false
end
Beginning and ending protocols of selected processes should be written in a way, that allows for these conditions to be met:
* Many readers should have simultaneous access to reading room.
* If in reading room there's writer, then no one else does not write or read.
* Each reader, which wishes for data read, at some point will read them.
* Each writer, which wishes for data modification, at some point will write such modifications.
There are variants to this problem:
* In reading room many readers can spend their time at once.
* Reading room may have limited capacity.
* Writers may have priority over readers (but then we'll resign from readers liveness).
* Readers may have priority over writers (but then we'll resign from writers liveness).
4. Five Philosophers.
This problem does not have practical analogies, unlike previous classic problems, but it very well illustrates problems happening when concurrent programs are made.
Five philosophers dine by round table. Before each there's plate. Between plates lie forks. In middle of the table there's serving dish with fish. Each philosopher thinks. When he gets hungry, he reaches for forks laying at his right and left side, then starts eating. When he's done eating, he puts forks away and again devotes himself to thinking.
Scheme of such philospoher is such then:
process Philospher (i: 0..4);
begin
repeat
think;
beginning_protocol;
eating;
ending_protocol;
until false
end;
Task is to write beginning and ending protocols, such as following conditions would be met:
* Only one philospoher ate with the same fork at the same time.
* Each of the philosophers ate only and always with two (and always those which lay near his plate) forks.
* No philosopher would die of starvation.
* Also we want that each of the philosophers would act the same way.
Solution to this riddle is to use waiter to let only four philosophers at the dining table at the same time, and let fifth wait. When philosopher is done eating, he leaves table and joins queue and can only return to table when waiter allows him to do this.
See also, if You wish: Basics of Concurrent Programming.
Subscribe to:
Posts (Atom)