Computers, Memory Pyramid & Code Size Optimization.

What is a Computer?

In Computer Sciences, Computer - by definition - is processor with memory and input/output devices. Any electronic device that has these is considered Computer. This includes Smartphones and many other tools.


Memory, Persistent or Transient.

There are two types of memory / pl: 'Są dwa rodzaje pamięci' /:
- Persistent / pl: 'Trwała' /,
- Transient / pl: 'Ulotna' /.

Persistent objects are those which continue to exist even after the program that created them has stopped running.

Transient objects cease to exist when program that created them stops.


Pyramid of Needs.

There are many types of memory, differing in price and speed of access.

Starting from the most expensive but fastest, there are:
- processor's registers,
- layers of the processor's cache (L1-L3, for example),
- RAM (Random Access Memory),
- persistent SSD/HDD storage.


Code Size optimization.

Smaller programs can be very quick in their execution.

When the whole program fits - for example - in L2 Processor's Cache, there's no need to reach RAM via BUS, so the code runs very quickly - as it's closer to the processor than RAM.


What if a Program doesn't fit in Transient Memory?

When a program needs to be executed, it needs to be loaded into the transient memory first.

However, Modern Operating Systems can send currently unused Program's parts & other Resources / for example: graphics image files, sound files and/or text files / from Transient Memory to Persistent SDD/HDD Memory and retrieve other Resources/Part(s) from Persistent Memory to Transient Memory / Usually from disk to RAM /.

/ pl: 'Współczesne Systemy Operacyjne mogą wysłać aktualnie niewykorzystywane części Oprogramowania i innych zasobów na dysk... i sprowadzić inne zasoby/części z pamięci trwałej do ulotnej, najczęściej do pamięci RAM' /.

Let's note, however, that loading/storing data in persistent memory is much slower than loading/storing from/to Transient RAM.

This is an automated operation in Modern Operating Systems, so programmers do not need to worry so much about that. Computers just slow down sometimes - and SSD/HDD becomes quite busy, when doing that.


This sometimes causes 'Flickering' / pl: 'Migotanie', 'Szamotanie' /, however. Code & Data is loaded/unloaded from/to persistent memory too slowly and can cause a Computer System to slow down or crash, as the Computing Resources run out. / Mostly CPU usage & Memory usage /.

So - in theory at least - a Computer can try to run larger programs than Computer has RAM.

Often it fails, but in theory this can work well.

Logic & Axioms.

'My Logic is based on different Axioms than Yours'.


There are many Logics, based on different Axioms.

/ EN: 'Axiom' = PL: 'Aksjomat' /.

Axioms are statements that are not proven, but assumed as true, taken on faith.

/ EN: 'assumption' = PL: 'założenie' /.

Depending on the Axioms used, Theorems can be proven or disproven, and the whole Mathematical & Logical Apparatus can be developed.


Basing on Boolean Algebra, double negation evaluates to confirmation, but in some languages - polish for example - double negation does not mean confirmation, it does mean emphasis on negation, giving negation more power.

/ EN: 'negation' = PL: 'zaprzeczenie' /,
/ EN: 'confirmation' = PL: 'potwierdzenie' /,
/ EN: 'emphasis' = PL: 'nacisk' /.


/ PL: 'nigdy nie zgodzę się na te warunki'. /

Therefore, Speech of the Art, Literature, can have the Logic based on different Axioms than Boolean Algebra.


We can also define the addition operation / it's an Axiom too / differently as well.

We can have an exception:

for example:

1+1 = 2, 1+2 = 3, 2+1 = 3, 1+3 = 4, 3+1 = 4, 2+2 = 5, 1+4 = 5, 4+1 = 5, 2+3 = 5, 3+2 = 5, 1+5 = 6, 5+1 = 6, 2+4 = 6, ...

We can also redefine the addition operation differently on the more general, more universal scale:

for example:

n+2 in our redefined addition operation is n+2+1 in classical addition operation.

By doing so, by changing Axioms, we just have revolutionized the Mathematics. ;)

Many different theorems apply now, but at least we know that we can make expression 2+2 = 5 to be evaluated as true in a certain Context - even if this brings more or less desired effects in process ;).


We are free to assume any Axioms we want, examples can be multiplied infinitely.


Which Logic is 'better' than other, then?

... it depends on the assumed Criteria, which might be Axioms as well.

The Enigma Cipher of WW2 & the Turing Machine.

Polish and British mathematicians were among the best of people who cracked the Adolph Hitler's cipher named Enigma, it happened during the World War 2nd.

Enigma breaking was hard, and the mathematicians were hunted by Germany's spies.

Enigma evolved, so parts of the cipher were to be cracked again and again. It was not about making an automaton once, and letting it work for the rest of WW2 ... but statistically it worked so the effort was continued.

Doing maths when time flew and lives were at stake.... so stressful. The Germany's spies added to the dangers & to the stress too.

Polish mathematicians had one of few of first computers ... it was nicknamed: 'Bomb', for it was so big invention. It increased efficiency of the enigma cipher cracking.

And there was Alan Turing's effort of course. He was a British scientist who laid foundation-theory of computer's construction.

His thinking is still present in computer sciences of modern days.

(We were taught about Languages & Automatons in Warsaw University when i was studying computer sciences. Turing's Machine was a part of this lecture).

> [ https://en.wikipedia.org/wiki/Turing_machine ].

The Turing Machine is programmed in a similar way to programming the Register Machine.

> [ https://en.wikipedia.org/wiki/Register_machine ].


--
Sources:

1. 'Cubits & Shrodinger's Cat. From Turing Machine to Quantum Computers' by John Gribbin.
(Polish Edition).

2. The internet (wikipedia & the ważniak mostly) and my own thinking.

3. My (unfinished because of health problems) education at Warsaw's University (Mathematics, Informatics & Mechanics Faculty).

Realtime Systems.

Introduction.

Real-Time systems are computer systems in which succesful task performance depends on two factors:
- succesful result of computation,
- exact deadline time in which task was performed.

When a task execution time exceeds it's time deadline, we can say that system failed.


Hard, Firm & Soft Real-time Systems.

In the 'Hard Real-time Systems' missing a deadline is a total system failure. Hard realtime systems are created when missing a deadline can result in hardware damage or costs lives or health.

In 'Firm Real-time Systems' infrequent deadline misses are tolerable, but may degrade the system's quality of service. The usefulness of a result is zero after its deadline, but no damage or personnel loss occurs.

In 'Soft Real-time Systems' the usefulness of a result degrades after its deadline, thereby degrading the system's quality of service.


Requirements.

For a computer system to meet criteria of the 'Hard Real-time System', requirements are:
- well understood & fast enough hardware,
- real-time operating system (for example: RTLinux),
- every software piece must adhere to the 'hard real-time requirements'; software must generate results in the deadline time.

Classic Concurrency Problems.

1. Mutual Exclusion.

Let's assume that two processess want to access critical section of code. They are acting as such:

process P;

begin
  while true do
  begin
    personal_affairs;
    begining_protocol;
    critical_section;
    ending_protocol;
  end
end;

Critical section is this program fragment, that can be executed by at most one process at once.

We assume that each process which enters critical section, will leave it in finite time.

See also: Petersen's Algorithm.

2. Producers & Consumers.

There are P>0 processes in system, which produce certain data, and K>0 processes which receive data from producers. Between producers and consumers there can be buffer with capacity B, whose task is balancing temporary differences during processes execution time. Processes that produce data we'll call producers, and processes receiving data --- consumers. Task is about synchronizing work of producers and consumers, such as that:

* Consumer would wait for data receive in situation when buffer is empty.
* Producer, when putting data in buffer would not overwrite data already written, but not received yet by consumer. It requires stopping temporarily producer in situation when there is no empty space in buffer.
* If many consumers wait for when data arrives in buffer, and if constantly new data is produced, then each waiting consumer will get something from buffer.
* Won't happen such situation, that certain consumer will wait infinitely for getting data, if data arrives in buffer constantly.
* If many producers wait for buffer's free space, and consumers constantly get something from buffer, then each of waiting producers will be able to put something into buffer. There won't happen situation such as certain producer will wait infinitely, if constantly something is taken from buffer.

There are variants to this problem:

* Buffer might be infinite.
* Cyclic buffer with finite space.
* No buffer at all.
* Many producers or only one.
* Many consumers or only one.
* Data might be produced and consumed at quicker rate (more than one unit at once).
* Data has to be read in writing order, or not.

Producers and consumers problem is abstraction of many situations existing in computer systems, for example: keyboard data write to buffer by keyboard device driver, and it's read by operating system.

3. Readers and Writers.

There are C>0 processes working in system. They read certain data. There are P>0 processes which write data. Processes writing data we'll call writers, processes reading data --- readers. Moment when processes have access to data, we'll call 'reading room visit' or 'reading room stay'.

Let's notice that many processes can read data at once. If someone wants to modify that data, then it's reasonable to block access to this data for all other processes for the time of read. It prevents read of inconsistent data (for example, partially modified data). Overall schema for processes work is such:

process Reader;

begin
  repeat
    personal_affairs;
    begining_reader_protocol;
    READ;
    ending_reader_protocol;
  until false
end

process Writer;

begin
  repeat
    personal_affairs;
    begining_writer_protocol;
    WRITE;
    ending_writer_protocol;
  until false
end

Beginning and ending protocols of selected processes should be written in a way, that allows for these conditions to be met:

* Many readers should have simultaneous access to reading room.
* If in reading room there's writer, then no one else does not write or read.
* Each reader, which wishes for data read, at some point will read them.
* Each writer, which wishes for data modification, at some point will write such modifications.

There are variants to this problem:

* In reading room many readers can spend their time at once.
* Reading room may have limited capacity.
* Writers may have priority over readers (but then we'll resign from readers liveness).
* Readers may have priority over writers (but then we'll resign from writers liveness).

4. Five Philosophers.

This problem does not have practical analogies, unlike previous classic problems, but it very well illustrates problems happening when concurrent programs are made.

Five philosophers dine by round table. Before each there's plate. Between plates lie forks. In middle of the table there's serving dish with fish. Each philosopher thinks. When he gets hungry, he reaches for forks laying at his right and left side, then starts eating. When he's done eating, he puts forks away and again devotes himself to thinking.

Scheme of such philospoher is such then:

process Philospher (i: 0..4);

begin
  repeat
    think;
    beginning_protocol;
    eating;
    ending_protocol;
  until false
end;

Task is to write beginning and ending protocols, such as following conditions would be met:

* Only one philospoher ate with the same fork at the same time.
* Each of the philosophers ate only and always with two (and always those which lay near his plate) forks.
* No philosopher would die of starvation.
* Also we want that each of the philosophers would act the same way.

Solution to this riddle is to use waiter to let only four philosophers at the dining table at the same time, and let fifth wait. When philosopher is done eating, he leaves table and joins queue and can only return to table when waiter allows him to do this.


See also, if You wish: Basics of Concurrent Programming.

Petersen's Algorithm.

It's solution to problem of synchronizing two processess (running programs) wanting to enter critical section of code (section that cannot be accessed by more than one process at the same time).

Using global variables (that indicate which process[-es] want to enter critical section, and which process waits) and simple programming instructions we can ensure that only one process enters critical section at given time.

Pseudocode:
var
  process1wants: boolean := false;
  process2wants: boolean := false;
  whoWaits: 1..2 := 1;

process P1;
begin
  while true do
  begin
    personalAffairs;
    process1wants := true;
    who_waits := 1;
    while process2wants and (whoWaits = 1) do {nothing};
    criticalSection;
    process1wants := false;
  end
end;

process P2;
begin
  while true do
  begin
    personalAffairs;
    process2wants := true;
    who_waits := 2;
    while process1wants and (whoWaits = 2) do
    criticalSection;
    process2wants := false;
  end
end;

Basics of Concurrent Programming.

Concurrent work is multiple works happening at the same time, processor time quants are divided among many processes.

Process is a running program.

Thread (lightweigh process, LWP) is object inside heavyweight process that has it's own control & that shares resources with other threads in the same process.


Critical Section.

Critical section is a code section that can be accessed by only one process or thread at the same time.

Critical section has uses for example in banking: we do not want data to be overwritten as it's written by other process, or read during writing.

Example pseudocode:

process P;

begin
  while true do
  begin
    personal_affairs;
    begining_protocol;
    critical_section;
    ending_protocol;
  end
end;

Readers & Writers.

In computer science, the readers-writers problem is an example of a common computing problem in concurrency.

Shared resource is abstracted as a Reading Room.

More than one Reader may be in a Reading Room, but if writer is in a Reading Room - no one else can be there.


An example algorithm for handling the readers-writers problem is as follows:

Reader's beginning protocol:
  reader waits (goes asleep), if there's writer in a reading room.

Writer's beginning protocol:
  writer waits (goes asleep), if there's someone in a reading room.

Reader's end protocol:
  if is last exitting person, then awakens & lets writer in - if writer waits.

Writer's end protocol:
  if readers wait, then awakens them all, otherwise, if writer waits, awakens one.


There should be a limit on maximum number of readers let in in a reading room, when writer(s) is/are awaiting.


See also: Petersen's Algorithm, Classic Concurrency Problems.