Thursday, 16 January 2025

Pentesting: Art of Reconaissance.

/ Work in Progress /

Introduction & OSINT.

Reconaissance is a part of pentesting process.

Information is the key to Strategy, and it happens before attack. In pentesting process, before we can exploit target system, we gather intelligence - both technical and non-technical - and build image of the target organization and it's systems. This will make it easier for us to perform attack, to exploit our target(s).

Types of reconnaissance
- Passive: Intelligence gathering without interaction
  - We collect information from public sources
- Active: Interaction with target
  - Sending requests and receiving responses
  - Login attempts
  - Ports scanning
    - Nmap
    - Nessus
    - Nikto
  - Interaction will leave tracks, for example in logfiles
  - Going beyond scope of test agreed with customer in the pre-engagement phase
    (see: PTES phases in previous article) of the pentest is law infringement

Enumeration
- Detailed analysis of services, machines and applications
- Identifying users by iterating
- Active technique

OSINT: Open Source Intelligence
- Also known as: 'White Intelligence'
- Three pillars of OSINT
  - Gathering intelligence about target organization
  - Obtaining information about infrastructure
  - Gathering other data
    - Sensitive information leaks
      - Passwords
      - Email addresses
      - Other sensitive data

When we gather intelligence about our target, at first we want to know the target organization.
- What it does?
- Whom it employs?
- Where it is located?

We build general image of the target.

Location:
- Large organization may be located in many cities or even nations
- On web pages and in social media we may find photos of the locations, both inside
  and outside
  - Sometimes on interior photos we can find sensitive informations
    - Employee list
    - WiFi password
    - Other
  - Outside photos can be used to find out where are cameras, which routes can be
    taken to enter the building
  - There were cases where hackers found camera model on image and used it to
    hack the camera, gaining unauthorized admin access
  - On interior photos sometimes computers can be seen, with running software
    This little bit of information can have serious consequences sometimes

Employees:
- Often on webpages we can find information about employees, which can be a starting
  point for social engineering attacks
- Data about employees can also be found in LinkedIn and in other social media
- We can find information about company itself
  - Whether target company is large or small
  - Whether employees are employed for long, or if there's high rotation of personnel
    - This may affect employees morale, and unhappy worker can be easy target for
      attacks
- It happens that on LinkedIn - for example - we want to make employer to contact us
  easier and we put information about our email address or telephone number in our
  profile
- Sometimes on organization's webpage there are documents to download. These files
  might contain sensitive informations
  - Often file formats contain metadata that can be extracted. This might
    contain used software, employee's name or even login, etc.
    This information may let us to profile our attack better
    - For example, if we notice that employee uses archaic version of
      excel program, we might succesfully use specially prepared xls
      file to perform an attack
    - There's tool called: 'exiftool' that can be used to extract metadata
- Colleecting contact information
  - Email addresses
  - Phone numbers
  - Hunter.io is one of many tools that allow us to find a person by his/her name
    and surname

Organization's workings (what/how it does)
- All kinds of sales results
- Operation reports
- Financial reports
- What organization offers
  - Services offered
  - What kinds of products are sold
    - How these products work
      - Manuals
      - Tutorials
      - You Tube films
- Activities of employees in social media
  - All kinds of company-related photos
    - 'Another day in office'
    - Corporate events

Discussion forums where support solves users' problems can be real treasury of informations.

There are services that allows to easily read reports, check connections between organizations or to check person to see connected organizations.
- For example: przeswietl.pl

All of the connected informations are useful, help us to get right image of target organization and to enhance our potential attack surface.


Reconnaissance: Domains & Subdomains.

/ To be continued /

Thursday, 9 January 2025

Introduction to Pentesting.

Introduction & Key Concepts.

What are Penetration Tests?
- Attack on System: Penetration Test is controlled attack on system, network or
  application
- Goal: Objective of Penetration Tests is checking hardiness to various types of
  attacks
- Role: Pentester acts as an attacker, while remaining cautious to not cause
  permanent damage to the system

Types of Penetration Tests:
- Internal: Tests taken from the internal network's perspective
  - Attack from inside: Pentester simulates attack coming from internal
    network of organization
  - Goal: Objective is to find vulnerabilities in internal
    infrastructure
  - Examples of vulnerabilities:
    - Unprotected servers
    - Weak passwords
    - Obsolete systems
- External: Tests taken from the external network's perspective
  - No access to internal network: Pentester attempts to gain access from
    outside of the Internet
  - Goal: Objective is to find security gaps
  - Vulnerable applications and services: Search for obsolete systems and
    configuration errors

Penetration Tests can also be categorized by knowledge we have about tested system(s)
- Black Box Testing:
  - No knowledge: Pentester has no prior knowledge about tested system
  - Black box: We approach system as closed, unknown structure
  - Information gathering: We must collect all needed informations using publicly
    available means
  - Creativity & skills: We rely on our creativity and on our skills in
    reconaissance and exploitation
- White Box Testing:
  - Full knowledge: We assume that we have full access to source code, documentation
    and architecture
  - Identifying vulnerabilities: Useful for finding security gaps in application's
    logic and in data flow
  - Deep analysis: We have more time for analysis of source code and architecture
- Grey Box Testing: Hybrid approach, something between Black and White Box Testing
  - Limited access: Pentester has limited access to the system
  - Inside threats: Identification of security gaps used by insiders
  - Red and Purple Teaming: Approaches that supplement the Grey Box Testing

Red Teaming:
- Simulation of realistic attack on organization
- Varied techniques: Using social engineering, phishing, exploits & other attacks
- Assessment of readiness: Checking real readiness for defense against threats,
  performing tests without prior warning

Purple Teaming:
- Cooperation: Red and Blue Teams cooperate
  - Red Team performs attacks using various techniques and shares findings with Blue
    Team
  - Blue Team uses knowledge gained from Red Team to enable adaptation to threat(s),
    to repel such attack(s)
- Constant learning: This approach promotes constant development
- Increasing security: Increases general security stance

Significance of Penetration Tests:
- Ensuring security: Penetration Tests are neccessary to provide organization with
  security & safety
- Varied perspectives: Varied types of tests provide varied informations, from varied
  perspectives
- Preparation for threats: Red and Purple Teaming exercises help organizations to
  prepare
- Constant development: Regular tests enable constant development of security
  mechanisms


Pentetration Testing Method.

When performing pentesting, it's convenient to use proven, standardized methods. One of such standard methodics is PTES: Penetration Testing Execution Standard. Every pentester develops his/her own methodics, but PTES is where we can start from.

PTES Phases: PTES divides pentesting process into 7 key stages
- Pre-engagement interactions:
  - Initial communication: Agreements between customer and the testing team
    - Time frames
    - Target system(s)
    - Limits
    - Signing contracts
      - Signing NDA contracts
      - Obtaining written permission for performing tests
  - Preparation: Planning and organization of testing
  - Agreements: Designating objectives and scope of pentesting
- Intelligence gathering
  - Collecting data about tested organization and it's systems
    - Customers
    - Partners
    - Employees
    - Used technologies
  - Analysis of public sources
    - Browsing internet pages
    - Checking social media
    - Checking public registries (WHOIS)
  - Using tools
    - Scanning network(s)
    - Searching for security gaps
    - Collecting technical data
  - Identifying targets
    - Services
    - Operating systems
    - Web applications
  - Objective: creating detailed image of test's target
- Threat modelling & vulnerability analysis
  - These two stages are often combined together, for practical reasons
  - Threat analysis: Designating threats and vulnerabilities
  - Automated scanning in search for vulnerabilities
  - Static & dynamic application code analysis
  - Testing validity of configuration
    - Application settings
    - Systems
    - Networks
  - Objective: Creating complex list of confirmed vulnerabilities that can be used in
    next stage
- Exploitation
  - Use of found vulnerabilities, to gain unauthorized access to certain resources or
    to raise (escalate) our privileges
  - If we succeed, we can make steps deeper into the infrastucture
    (so called: 'lateral movement')
  - Creation / adjustment of exploits
  - We must be careful to not disrupt working of customer's production system(s)
  - Publicly available exploits sometimes pose threat to correct performance of
    customer's production system(s). Code of such exploits has to be
    analysed before use - in an attempt to predict it's effect(s) on
    customer's production system(s)
- Post-exploitation
  - Re-enumeration: Analysis of new privileges and possibilities
  - Confidential data identification: Discovery of new, critical resources
  - Maitaining access: Testing means for maitaining control
- Reporting
  - Notations
    - Recording found vulnerabilities
    - Passwords
    - Keys
    - Usernames
  - Summary: Compiling all of important data for the report
  - Detailed report: Description of methods and recommendations, even up to 100 pages
  - Report should be created in parallel with performed penetration tests, to make it
    easier to summarize findings
  - Elements of a good report
    - Executive summary: summary for non-technical personnel, associated with
      business. Often presented in a short, condensed, 1-2 pages report
      - Summary for the management staff
      - Technical summary of found vulnerabilities: Detailed, understandable for
        everyone description of found security gaps
      - Vulnerability assessment & recommendations: Risk assessment &
        corrective recommendations
      - Evidence: Report should include evidence that confirms found problems
    - Technical description of vulnerabilities
      - Detailed description: Detailed description of identified vulnerabilities,
        including all of the steps neccessary to reproduce the vulnerability
      - Impact: Presentation of potential impact for each of found vulnerabilities
      - Vulnerability assessment: For each of vulnerabilities we should include it's
        assessment according to the CVSS 3.1, categorization, and estimated
        impact on business
      - Recommendations: Recommendations regarding vulnerability removal, correction
        of errors & improvement of security
      - Repair actions: Prepared plan for repairing, based on tests report
      - Evidence
        - Screenshots
        - Logfiles
        - Confidential data should be masked/partially hidden (passwords for example)
  - Importance of good report
    - Key result: Good report allows for understanding and resolving identified
      security problems
    - Quality of work: Report speaks about quality of pentester's work
    - Closure of the Pentest: Report finishes the last stage of pentest, and closes
      the last stage of the PTES methodics

PTES includes detailed guidelines and checklists for each of pentest's stages, helping pentesters to perform complex & methodical penetration tests.

PTES's elasticity:
- Adjustable for context: PTES provides elastic guidelines, that depend on tested
  system
- Team cooperation: Common methodics makes it easier for pentesting teams to cooperate
- Overlapping approaches: PTES & OSCP methodics have many of common elements

Gentleness at pentester's work:
- Cooperation, not rivalry: Objective is increasing security, and NOT proving
  one's superiority or showing off
- Empathy & understanding: Let's remember that other workers also want to increase
  the security
- Common goal: We all play to the common goal, we want to increase security together
- Tone of speech: Avoid only pointing at errors, also appreciate the efforts of the
  whole team
- Building cooperation: Describe good sides of realized solutions in the report
- Communication & 'soft skills': Also important in the pentester's work


Where to hack legally?

CTF Platforms (Capture the flag):

Mostly for testing/honing skills, but sometimes there are financial rewards.

- Hack The Box: https://www.hackthebox.com
- VulnHub: https://www.vulnhub.com
- TryHackMe: https://tryhackme.com

Bug Bounty Platforms:

There are financial rewards, but competition is big.

- HackerOne: https://www.hackerone.com
- Intigriti: https://www.intigriti.com
- BugCrowd: https://www.bugcrowd.com

Sometimes found vulnerabilities are reejected (for example: when someone else found the vulnerability earlier), not every time one gets paid.


Building Hacker's Mindset.

For a pentester, it's worthwhile to develop Hacker's Mindset, to develop following 9 of mind's qualities:

Desire for constant development
- Dynamic domain: New threats and fast development of technology
- Passion for learning: Specialist has to follow newest trends
- Beginning of the adventure: This article is just a beginning
- Constant development: Don't stop here

Think like a hacker
- Learn attack methods
- Offensive approach: Learn to think as attacker, to defend better
- Use gained knowledge to protect the systems

Be curious
- Learn various topics in depth, don't stop at shallow knowledge
- Various approaches: Try various methods and ways for problem solving
- Creatitivity: There's no single and true way, be open for new ideas

Analytical thinking
- Break down issues into prime factors, analyse information to find vulnerabilities
- Connect seemingly unrelated elements
- Predict where something can be broken

Communication skills
- Cooperate with the rest of the team
- Adapt the language to the recipient's level
- Constructive critic: Convey comments in a sensitive manner

Resistance to stress:
- Keep calm: Stay focused when under pressure
- Plan a'priori: Plan all steps to be taken in advance
- Keep tools ready
- Train personnel in advance

Ethics:
- Be cautious with access to sensitive information
- Stay within agreed scope of the pentest
- Remember to be responsible with regard of users' security

Don't give up
- Be determined and persistent
- Don't give up when difficulties arise
- Take a break: rest for a while then return to problem
- Keep trying until you succeed

Solve problems:
- Try solving problems on your own
- Slow down, think, and try to analyse problem again
- Think creatively: Try to find non-obvious solutions. That's hacking after all

Thursday, 19 December 2024

Foundations of Cyber Security.

Introduction.

Each year brings more and more of software and devices connected to the Internet. As more of such are connected, system vulnerabilities increase in numbers as well.

There's deficit in numbers of cybersecurity professionals (in Poland alone, there's need for over 15 000 cybersecurity experts), and it seems that this deficit will increase in numbers as years pass.


Red Team vs. Blue Team.

'Red Team Hackers', 'Ethical Hackers', also called: 'Pentesters' focus on finding vulnerabilities in organization's systems, then report and help Blue Team to 'patch the holes'.

Blue Team are Administrators who are installing, configuring, and monitoring antivirus software, intrusion detection systems, and other protective mechanisms on these devices.

This article concerns more about theory and is mostly of interrest for future Blue Team members. It touches only basics, however. Blue Team aspirants should deepen the knowledge using other means as well.

Red Team members are meant to cooperate & communicate with Blue Team, hence why this article should be useful for them as well.


CIA Triad.

C - Confidentiality
    - Ensuring that informations are available only for authorized personnel
I - Integrity
    - Protecting data from unauthorized modification or deletion
A - Availability
    - Ensuring that informations are available for authorized personnel

Methods for Ensuring Confidentiality:
- Security Policy (Categorizing Data as either Confidential or Public,
  and ensuring that only authorized personnel has access to Confidential Data)
- Encryption
- Access Control (Physical and Multifactored Access Control)

Securing a device often comes at the cost of convenience of use. Protected laptop computer, for example, should be still useful and use-conveniet for authorized personnel. So, in practice, security means should not be too excessive. We should not request, for example, 128-characters-long random passwords from users.

Meaning of Integrity:
- Data should be protected from unauthorized modification
- Example attacks on Data Integrity:
  - Modifying company's payments lists
  - Modifying company's webpage conent files
- Disrupting Integrity might lead to disastrous consequences for a company

Meaning of Availability & Methods for Ensuring Availability:
- Ensuring continuity of work of systems and applications
- Redundancy (extra resources for case of malfunctions)
- Keeping backups (for swift restoration of data and systems in case of malfunction)

Examples & Solutions for Availability Violations:
- DDoS attacks (Resources Overload might cause troubles in accessibility)
- Viral Marketing (Sudden increase in Web Traffic might lead to overloading of
  server resources)
- Solutions:
  - Scalability
  - Load Balancing
  - Firewalls
  - Monitoring

Additional characteristics of CIA model:
- Authentication & Authorization (verifying user's identity and privileges)
- Non-repudiation (provides proof of the origin, authenticity and integrity of data.
  It provides assurance to the sender that its message was delivered, as well as proof
  of the sender's identity to the recipient)
  This way, neither party can deny that a message was sent, received and processed)
- Robustness (ability of a computer system to cope with errors during execution
  and cope with erroneous input)
- Compliance (fulfilling legal and regulatory requirements)
- Privacy (protecting user privacy and personal data)


Cybersecurity as Process - CSF2.0

Overview:
- Cybersecurity should be treated as continuous process, not as single, one-time,
  100% complete solution
- Every day new attacks & hacking techniques appear. Hacking tools are under constant
  development, with time more advanced and more sophisticated hacking tools appear
- There's need for constant updating of security strategies
- Organizations keep deploying new technological solutions. New technologies might
  introduce new vulnerabilities, new security holes

Being ready for incidents:
- Preparation (developing incident reaction plans)
- Detection (swift identification of potential threats)
- Reaction (immediate reaction to appearing incidents)

Process approach to cybersecurity can ensure readiness for action at any moment.

Process approach to cybersecurity can make compliance with legal and industry requirements easier.

Process approach in practice:
- Planning (developing strategies and objectives of cybersecurity)
- Deployment (implementing planned actions & controls)
- Assesment (efficiency analysis of deployed solutions)
- Perfection (constant refinement of security practices)
- Continuity & cyclicity (keep repeating above solutions in cycles)

Elasticity & Adaptation:
- Fast adaptation to appearing threats
- Adaptation to changing technologies & IT architecture
- Adaptation to changing business requirments, to changing organization needs

Integration with business processes (cybersecurity should not be treated as a
separate concern, should a part of business processes):
- Risk Management (integrating cybersecurity with overall risk management)
- Products Development (take cybersecurity into consideration with products'
  lifecycles)
- Customer Service (integrating cybersecurity with customer support practices)

Cyber Security Framework 2.0 (CSF2.0):
- Created & Developed by NIST (National Institute of Standards & Technology in USA)
- Objective: helps with Risk Management in Cybersecurity
- Universality: can be tailored for small or large companies & organizations

CSF Elasticity:
- Elastic: CSF is an elastic tool, not stiff regulation or standard
- Adjustment: can be tailored for specific organization needs
- Pointers: generic approach, with possibility of custom implementation

CSF2.0 Components:
- Core (center of framework, with key functionalities)
- Profile (description of current and target cybersecurity approach)
- Tiers (characteristics of rigors of actions related with risk management)

CSF2.0 Core:
Govern:
- Describes strategy of Risk Management in organization
- 'Surrounds' and manages five other CSF2.0 components
- Includes determining roles & responsibilites, who is responsible for what
- Manages supply chain(s)
- Sets policies related with cybersecurity

Identify (understanding of what we want to protect):
- Understanding (increasing understanding of current risks in cybersecurity)
- Resources Management (identification & management of organization's resources)
- Risk Assesment (analysis & evaluation of potential threats)

Protect:
- Security Tools (deploying proper security tools for risk management)
- Access Control (managing authentication & authorization, and raising
  users' cybersecurity awareness)
- Data Security (protecting organization's sensitive informations)

Detect:
- Monitoring (constant monitoring of systems and networks)
- Analysis (analysis of incidents & anomalies in realtime)
- Alerts (generating alerts of potential security incidents)

Respond:
- Reaction Plan (preparation and deployment of incidents reaction plans)
- Incident Analysis (detailed analysis of detected cybersecurity incident)
- Softening Effects (actions meant for minimizing effects of security incident)
- also: Reporting & Communication

Recover:
- Recovery Plan (executing plan meant for restoring normal functioning)
- Communication (informing interrested parties about recovery processes)
- Post-Incident Analysis (drawing conclusions & refining processes)

CSF2.0 Profiles:
- Current Profile (description of current organization's cybersecurity state)
- Target Profile (description of desired cybersecurity state to achieve)

CSF2.0 Tiers:
Tiers determine how organization approaches and manages risks on the whole, how much of resources are spent for which risks.

Organization should consider how dangerous which risks are, resources available, and possibility of certain solution deployments.

Choice of Tiers for risks depends on organization. Not always higher tiers mean better security. Organization should choose which tiers to assign for what risks, depending on organization's resources and current needs.

Integrating CSF2.0 with overall risk management:
- Balance (treating cybersecurity risks as equal with other risks)
- Integration (including cybersecurity in overall risk management)
- Development (constant development of risk management processes)

Links:
- NIST CSF2.0
- Implementation examples
- Quick Start guides


Vulnerabilities and Metrics - CVSS.

Vulnerability:
- Vulnerability is weakness or an application error in system or process
- Vulnerability can be used to perform an attack
- Vulnerability is threat to application security or to system security

Causes of Vulnerability:
- Configuration Error(s)
- Errors in Design, in application's architecture
- Lacks in security updates
- Hasty application debugging, programming shortcuts

Human factors in Cybersecurity:
- Weak passwords
- Phishing (unaware users might reveal confidential informations)
- Lack of cybersecurity courses, lack of threat awareness
- Short, interresting courses help to pass the knowledge

Difference between error and vulnerablitiy:
- Error is unintended malfunction in software
- Vulnerability is an error that can be used to perform an attack
  not every error is vulnerability
- Vulerability allows for modification in application's performance,
  in a way unintended by it's creator(s)

What happens when vulnerability is found:
- Reporting vulnerability to application's creator(s)
- Vulnerabilities catalogue (CVE - Common Vulnerabilities and Exposure)
- Responsible revealing
  Investigators cooperate with app delivery to patch the security gap(s)

Introduction to CVE vulnerabilities catalogue (Common Vulnerabilities and Exposure):
- Created by MITRE in 1999
- Goal: Standardization of identification of known vulnerabilities
  Each vulnerability has standardized identifier: CVE - year - ID number
- Informations exchange: Allows for efficient informations exchange about
  vulnerabilities

Introduction to CVSS standard (Common Vulnerabilities Scoring System):
- CVSS allows for classification of vulnerabilites, from low priority to critical
  vulnerabilities
- CVSS is a standard method for classification of vulnerabilities' threat levels
- Prioritization: helps to determine which vulnerabilities require more urgent
  attention
- Severity Score (for example: 8.8 High)
- Vector:
  - Attack Vector (for example: Network)
  - Attack Complexity (for example: Low)
  - Privileges Required (for example: Low)
  - User Interaction (for example: None)
  - Scope (for example: Unchanged)
  - Confidentiality (for example: High)
  - Integrity (for example: High)
  - Availability (for example: High)

Reaction to high and critical vulnerabilities:
- Immediate software update
- Reconfiguration: reconfiguration of the system, deployment of solution means
- Rescanning: efficiency analysis of deployed solution means

Tools for automatic scanning of vulnerabilities (as Nessus for example):
- Detected vulnerabilities are prioritized (Highest Score vulnerabilities should be
  handled earlier)
- Chaining vulnerabilities: less important security gaps can be combined into
  dangerous threats

Holistic approach to vulnerabilities management:
- Overall assesment: considering potential interactions between vulnerabilities
- Grouping associated vulnerabilities: considering overall impact on security
- Knowledge of architecture: knowing data flow between components
- Scanning & patching of vulnerabilities on regular basis

Links:
- CVE Catalogue (MITRE): https://www.cve.org/
- Vulnerabilities Database (NIST): https://nvd.nist.gov/vuln
- CVE-related statistics: https://www.cvedetails.com/
- CVSS (3.1) Calculator: https://nvd.nist.gov/vuln-metrics/cvss/v3-calculator


Threat and Threats Modelling.

Threat is potential event, in which vulnerability might be used to cause harm.

Examples of threats:
- Unauthorized access
- Data stealing
- DDoS attacks
- Infections with malware

Threats can endanger Confidentiality, Integrity and/or Availability of data, application(s) and/or system(s).

Sources of Threats: Threats can originate from within or from outside organization.

Threat modelling:
- Understanding threats: analysis of potential threats and weak points
- Evaluation of potential effect on organization
- Proactive actions: creation/developmentt of strategy for protecting against threats
- Objectives of attackers: understaning of what pottential attacker wants to achieve
- Motivation analysis: what can cause someone to attack
- Resources: identification of attractive resources for attacker

In other words, threat modelling is looking at a system from attacker's perspective, with goal of predicting pottential attack(s) vector(s)

Difference between threat modelling & vulnerability assessment
- Vulnerability assessment: scanning system(s) with goal of discovering known
  vulnerabilities
- Threat modelling: analysis of potential threats, and means of protection
- Perspective:
  - Vulnerability assessment: from defense perspective
  - Threat modelling: from attacker's perspective
    - System decomposition: analysing components for weak points
    - Brainstorming: discussing about potential threats for our application

OWASP Cheat Sheets: help to approach security threat(s) in condensed form.

Involving team in threat modelling process:
- Cooperation: Include people from various areas into threat modelling session
- Different perspectives: discuss security concerns in larger group
- Effectivity: arrange for focused session with programmers

Four key questions to answer in threat modelling:
- What we're working at?
  - Decomposing system(s): split system into smaller, easier to analyse parts; analyse
    each of components separately to understand them better
  - Understanding system: know components, functions, data flow & trust limits
- What can go wrong?
  - Identifying threats: considering potential problems for each of of system's
    components
  - The STRIDE model can be used to go through each of threats categories in a
    systematic manner
  - Group analysis: we identify & discuss potential threat(s) together
- What we intend to do with it?
  - Risk reduction: we aim to reducte the risk(s) to acceptable level(s)
  - Architectural changes: we introduce modification(s) in system's architecture
  - Security functions: we add concrete functions that increase security
  - Processes correction: Increasing quality of processes, to reduce threats
- Did we do good enough job so far?
  - Effectivity assessment: checking whether deployed solution(s) effectively solve
    identified threats
  - New threats: we should make sure whether new threats appear because of introduced
    changes
  - Continuous process: threat modelling should be repeated constantly, as a
    continuous process

STRIDE Model:

STRIDE Model analysis example (in this case we're considering user interface issues):

Spoofing: pretending to be someone else
- Identity verification
- Authorization: weak authorization algorithms might lead to spoofing
- Threat: insufficient identity verification is the key threat

Tampering: manipulation
- Data manipulation: can data sent via user interface can be manipulated?
- Validation: potential problem is lack of server-side data validation
- Integrity protection: data-integrity protections should be handled well enough

Repudiation: denying involvement
- Denying action: user may try to deny that he/she performed a specific action in
  our system
- Insufficient registration: lacks in proper actions logging can make proving
  of responsibility for performing an action more difficult
- Protecting from repudiation: system should be fitted with efficient
  users actions registration systems.

Information Disclosure: revealing confidential informations
- Information reveal risk: user interface can reveal confidential data
- Improperly handled errors: can cause sensitive informations to be revealed
- Improper encryption: data can be captured during transmission process

Denial of Service: Refusing access
- Vulnerability to DoS attacks: can user interface be used to perform DoS attacks?
- No limits: Without requests limits, the system can be overloaded when traffic is
  high

Elevation of Privilege: raising privileges
- Security holes like XSS can allow to perform actions with attacked user's privileges
- Threat to system: can user gain higher privileges through the user interface?
- Security holes may allow attacker to raise his/her privileges

By performing STRIDE Model Analysis for each of components in attacked system, we can make a complex image of potential threats. This allows to design proper security solutions for protected systems.

Strong points of STRIDE Model use:
- Systematic analysis: ensures complex research of security concerns
- Methodical approach: considers every threat category, for each of analysed elements
- Minimizing risks: lessens the risk of overlooking important security gaps

STRIDE Model's limits:
- Generic model: STRIDE is abstract model, without concrete guidelines
- Elasticity: allows for high elasticity when thinking about threats
- Knowledge: requires supplementing with technical knowledge about concrete
  technologies

Documenting threats:
1. Threat name
2. Description
3. Objective: what is the goal of the attacker?
4. Attack technique:
5. Required privileges
6. Impact on the customer/business (who ordered our protected application)
7. Security means to deploy

Risk assessment:

Severity of risk depends on two factors:
- Impact on the customer/business (who ordered our protected application)
- Probability of attack's occurance

Prioritizing threats:
- High Risk: immediate attention needed (for example: SQL Injection; insufficient
  authorization)
- Mid & Low Risk: to be solved later
- Repeated regularly: threat modelling process should be repeated as the protected
  system develops and gains in complexity

Other frameworks for threat modelling (STRIDE's competition):
- PASTA
- Trike
- OCTAVE
- VAST

Free tools for threat modelling:
- Microsoft Threat Modelling Tool
- OWASP Threat Dragon

Links:
- Threat Modelling Process (OWASP)


Risk Management.

Risk in the Cybersecurity context:
- Risk is probability of occurance of undesirable event(s)
- Two main factors:
  - Probability of occurance
  - Impact on our organization
- Risk's effects:
  - Financial losses
  - Physical damage
  - Loss of trust
  - Negative PR
- Benefits of definition:
  - Precise grading and prioritization

DREAD model (created & developed by Microsoft):
- Damage potential: how severe damage an attack can cause?
- Reproductibility: how easy is to repeat the attack?
- Exploitability: how easy is to perform the attack?
- Affected users: how many users are affected by the attack?
- Discoverability: how easy is to discover the security gap?

DREAD model is meant to help us to more precisely analyse the potential threats, and allows for quantitative analysis which is very useful in the Risk Management process. The risk can be measured and described with a number, which allows for easier and more precise risks comparisons and helps in the Risk Management.

DREAD model validity:
- Allows for quantitative analysis of potential threats
- Makes Risk Management easier by expressing risks in form of numbers
- Assessment of each factor in the 1-10 scale depends on experience of person who
  assesses the risk

Hints for assigning numbers to risks:

1. Damage potential scale:
   - Minimal damage: 1-2
   - Low damage: 3-4
   - Moderate damage: 5-6
   - Serious damage: 7-8
   - Catastrophic damage: 9-10
2. Reproductibility:
   - Very difficult to repeat: 1-2
   - Difficult to repeat: 3-4
   - Moderately difficult to repeat: 5-6
   - Easy to repeat: 7-8
   - Very easy to repeat: 9-10
3. Exploitability:
   - Very difficult to perform attack: 1-2
     - Requires expert knowledge and nonstandard tools, or huge amount of resources
   - Difficult to perform an attack: 3-4
     - Requires advanced knowledge and specific tools
   - Moderately difficult to perform an attack: 5-6
     - Requires some of technical knowledge, and available tools
   - Easy to perform an attack: 7-8
     - Only basic knowledge is required, attack can be performed using standard,
       generic tools
   - Very easy to peform an attack: 9-10
     - Require practically no qualifications from attacker
     - Often these attacks can be automated
4. Affected users (how many users can be affected by the attack):
   - Very small part (less than 1%): 1-2
   - Small part (1%-5% of users): 3-4
   - Significiant part (5%-25% of users): 5-6
   - Major part of of users (25%-75% of users): 7-8
   - Most of users (more than 75% of users): 9-10
5. Discoverability:
   - Very difficult to discover: 1-2
     - Requires specialised knowledge and 'deep' testing
   - Difficult to discover: 3-4
     - Requires advanced knowledge and 'directed' testing
   - Moderately difficult to discover: 5-6
     - Requires some of 'directed' testing
     - Probably will be discovered during 'routine' testing
   - Very easy to discover: 9-10
     - Easy to guess by merely interacting with attacked system

The above hints are just generic tips, should be adjusted for the needs of organization we protect and for specific context/situation in which we're in. Key is consequence and clear communication about criteria in the whole team. Whole team should use consistent, unified scale.

It is also important to review and update risk assessments for the organization and/or system(s) as the organization we protect grows and develops.

After assigning numbers to risks, we add the number and consult the result with following table to get the Risk Level, according to the DREAD model:

The DREAD model is only one of many tools, and has it's weaknesses/limits too. It doesn't consider probability of attack, for example. It's still useful tool for risks prioritization and communication.

After assessing the Risks Levels, we should answer the following question: what we do with it? how to manage it?

Risk Mitigation
- Reducing probability or impact
- Lessen effects or risk

Risk Management:
- Risk Avoidance
  - We eliminate actions/processes or resources that make our organization
    vulnerable to unneeded risk(s)
- Risk Reduction
  - Example: Deploying Multi-Factor Authentication
  - Example: Encrypting Laptop's Hard Disk (Mass Storage)
    (Even if the laptop is stolen, important informations won't leak)
- Risk Transferrance
  - Insurance
  - Moving service to the Cloud
    (Risk is transferred to the Cloud services provider)
  - SLA contracts
    (Defining responsibilities division in the contract)
- Accepting Risk
  - Deliberate decision
  - In everyday life we accept small risks
  - Sometimes accepting risk has more benefits than troubles and costs attached


Risk Metrics.

With Risks there are metrics associated. These metrics help to calculate potential lossess associated with risks, and benefits and costs of investments that reduce these risks.

SLE - Single Loss Expectancy:
- Value of losses associated with a single incident
- Calculated as follows:

SLE = resource's value * exposure factor


ARO - Annual Rate of Occurence: how many times per year, an attack occurs

ALE - Annual Loss Expectancy:
- Estimates yearly loss associated with risk
- Calculated as follows:

ALE = SLE * ARO


AAL - Annual Avoidance of Loss:
- Metric for potential yearly losses that can be avoided
- Example: deployment of Multi-Factor Authentication can reduce losses by 80%
- Interpretation: for assumed value of 100 000 PLN yearly loss associated with
  incident(s), with 80% loss reduction factor we can avoid losing an amount
  of 80 000 PLN per year
- Calculated as follows:

AAL = ALE * reduction factor

(in an above example: AAL = 100 000 PLN * 80% = 80 000 PLN)


ROI - Return of Investment:
- Metric for investment profiatbility, describes benefits/costs ratio
- High ROI means that investment is business-justified
- Calculated as follows:

ROI = (AAL - investment cost) / investment cost


Importance of Risk Metrics:
- Metrics help to justify actions before the management staff
- Losses assessment makes it easier to plan the budget
- Metrics help to designate benefits from the actions taken

Business issues might seem boring for a cybersecurity specialist, but these improve communication with business, with management staff. With business metrics, informed decisions can be taken by the business staff. Professional cybersecurity personnel should not ignore business matters as well.


Attack & Incident Reaction Plans.

When we consider an attack, we should coniser WHEN it happens, not WHETHER it happens.

What is an Attack in Cybersecurity?
- Attack is a moment when potential threat becomes reality, when it 'materializes'
- Attacker uses vulnerability, and causes security incident
- Theory becomes practice - organization needs to react
- Swift steps are neccessary to deal with incident

In CSF2.0 Core we've considered 3 phases of incident's response: Detect, Respond & Recover. CSF2.0 does not giver us, however, any detailed solutions associated with reaction plan. That's where SANS Incident Response Cycle framework might come handy.

SANS Incident Response Process:
1. Preparation
  - Roles & Responsibilities designation
  - Setting up the the efficient communication channels to use in case of incident
  - Setting up the procedures for dealing with incident
  - Creation and training the incident reaction teams
    - Incident reaction teams should consist of people with varied skills, including:
      - IT specialists
      - Security Experts
      - Lawyers
      - PR Representatives
    - Regular training is the key for the team to stay up to date with the most recent
      threats, and to efficiently cooperate under the pressure of time
  - Providing neccessary tools & resources
    - Security Monitoring Systems: for incident detection & reaction
    - Tools for Forensic Analysis: for investigating attack's causes & effects
    - Backup creation tools: for quick data & systems recovery
    - Dedicated testing environment: for testing & analysis of incidents in safe
      conditions
2. Identification
  - Monitoring of systems & networks
    - Detecting anomalies:
      - Unusual network traffic
      - Failed login attempts
      - Unauthorized files modifications
    - Identifying deviations from norms:
      - Anything that deviates from normal functioning
  - Analysis of alerts & indicators of compromise
    - Analysis of alerts and idicators of compromise (IoC) may help to identify an
      incident
    - Checking whether alarm is false or not, is the key
    - Attack traces (IoC) may confirm that attack actually happened
      - IoC types:
        - IP Addresses
        - Domains
        - File hashes
        - File names
        - Character sequences
        - Registry keys
        - Network traffic patterns
      - IoC Changeability: IoC quickly lose relevance, because attackers often change
        tools & infrastructure
      - Modern approach: Combining IoC with behavioral analysis & machine learning
        can result in better threats detection
  - Determining reach & impact: once we've confirmed that attack happens, we should
    determine it's range & impact
    - Quick designation of affected systems
    - Identification of compromised data
    - Assessment of potential impact on organization
3. Containment: preventiont of spread of damage, as quickly as possible
  - When whole team is trained and ready (see phase: 1. Preparation), the quick
    containment can be real
  - Isolation: disconnecting infected systems from network
  - Blocking: blocking malicious accounts in applications
  - Limiting: limiting spreading of threat
  - Deployment of temporary corrective measures
    - Quick, temporary bugfixes in applications that can stop certain attack vectors
    - Firewall blocking rules that blocks specific network traffic or certain
      IP Adresses
4. Eradication: removal of the cause(s) of attack
  - Requires 'Deep' analysis of the incident, for identification of the threat's
    source(s)
  - Removal of all related security gaps, not only symptoms used in the attack
    - Removing ALL of the malicious code detected during the analysis
    - The cause of vulnerability that enabled the attack needs to be identified and
      fixed, for example by:
      - Reinstallation
      - Reconfiguration
      - Patching the application's code
5. Recovery: restoring systems to normal functioning, but without too much of haste -
   to not allow systems to be re-infected
  - Before we go to the recovery phase, we must positively confirm that attack has
    been stopped
  - Prioritization:
    - Designating the most important systems to recover first
    - Prioritization by business priorities: production line first, then HR systems
  - Verifying backups
    - We should make sure that backup we restore from is not infected
    - We should 'deeply' perform antivirus scanning
    - We should review security patches and test vulnerabilities
    - We should monitor systems after restoration
    - We should make sure that the compromise is not repeated again
6. Lessons Learned:
  - Incident analysis, for improvement of processes, and for increasing team skills
  - Identification of good practices and areas for improvement in the reaction plan

Reaction to incidents is continuous process, and not a single exercise.

Every incident is an opportunity for learning and development, requires constant readiness.

SIEM - Security Information and Event Management.

An example of SIEM use in practice:
- SIEM aggregates logfiles from various systems & networks, enabling analysis of
  suspicious events
- For example: SIEM might detect that user account writes large amount of data during
  unusual hours
- Security analyst investigates this and confirms that account has been hijacked and
  executes formally reaction to incident


Who is Attacker & Attack Types.

Threat Actors & Motivations:
- Enemy Actors: Enemy persons and groups that pose threat to our organization
- Motivations: Various reasons of why attackers take actions
- Attack vectors: Means by which attackers gain access to our systems

Regarding the Cybersecurity Myths:
- Every company can be attack's victim, no matter size or industry
- Attacks may be fully automated, or precisely planned and executed

Attackers' Types & Motivations:
- Script Kiddies: Young attackers
  - Use pre-made scripts & exploits
  - Low skills: don't know how to hide traces, easy to identify
  - Motivations:
    - Malice
    - Nastiness
    - Desire to impress
    - A joke
- Organized Cribe: Dominant group of attackers
  - Motivation:
    - Ggetting financial profits
  - Popular attacks:
    - Ransomware
    - Stealing credit card informations
  - Profit scale:
    - Estimated profits from ransomware attack: 1 Billion USD
  - Discretion:
    - Victims often don't disclose the ransom paid
- Hacktivists:
  - Motivation:
    - Social ideas
    - Political agenda
    - Not interrested in financial gains
  - Attacks' objectives:
    - Protests against corporations, governments or politicians
    - Revealing informations
  - Perceptions:
    - Acting in the name of the 'greater good', according to their own beliefs
- Insider threats:
  - Unhappy employee
    - Can cause damage if has too broad privileges
  - Greedy & unethical employee
    - May want to steal data to gain financial profits
  - Unaware employee
    - Can randomly open infected file or email attachment
- Advanced Persistent Threat (APT)
  - Directed campaigns:
    - APT target large companies, critical infrastructure and governments
  - Used security gaps:
    - Attackers use unknown to public security gaps
  - Long-term actions:
    - APT can hide in victim's networks for years
  - Political motivations
    - APT often operate on government request, for espionage

Attack Vectors: paths & methods used by attackers to gain unauthorized access to networks, applications, computers or data
- Phishing: Attacks that depend on cheating/deceiving users
  - Attackers use emails, SMS, or falsified webpages to extort data, especially logins
    and passwords
- Malware: Malicious software that infects systems
  - Malware types
    - Viruses
    - Worms
    - Trojans
    - Ransomware
    - Spyware
  - Spreads through:
    - Infected attachments
    - Links
    - Webpages
  - Potential damage:
    - Malware can cause serious damage
- Man-in-the-Middle: Capturing communication between webpages/applications
  - Eavesdropping communication between two pages/applications, often without their
    knowlege
  - Capturing communication: Attacker may gain control over the communication
- Attacks on passwords: Attempts to steal or guess the passwords
  - Brute force attacks: attempt to guess the password by trying various combinations
  - Dictionary attacks: using lists of popular passwords to gain acccess
  - Credential stuffing: attempt to use stolen credentials on varied services
- DDoS: Distributed denial of service: attack that attempts to overload systems
  or networks
- Social Engineering
  - Psychological manipulation: use of human psychology to gain confidential
    informations
  - Key element of phishing
- SQL Injection: Attacks performed by injecting malicious SQL code
  - Attempt to gain unauthorized access to data
  - Attacks directed on web applications' vulnerabilities
- Cross-Site Scripting (XSS)
  - Publishing malicious scripts on legal webpages
  - Unaware execution of a script can lead to stealing session's cookies
- Attacks on Supply Chain(s)
  - Compromising a supplier
    - Attacker(s) compromise trusted supplier of hardware or software
    - Attackers replace files or infect hardware
    - User installs the infected software, without knowledge or suspicion
- Attacks on Wireless Networks
  - Attackers might eavesdrop the unprotected network traffic
  - Attackers can create false access points
- Attacks on IoT (Internet of Things)
  - New domain for hackers
  - Many of IoT devices has insufficient protection
  - Hacker might steal control over IoT device(s)
- Reverse Engineering
  - Hackers decompile and analyse software, looking for security gaps
- Memory-related Security Gaps
  - Buffer Overflow
    - Memory overflow in vulnerable application might lead to malicious code execution
- Attacks on Cloud Infrastructure
  - Cloud servicesa are complex systems
  - Attackers use configuration errors and other security gaps in cloud services
- Attacks on Mobile Applications
  - Personal data: Smartphones are real treasury of personal informations
  - Mobile Applications security gaps: Attackers search for weak points in mobile
    applications, with an objective to gain access to sensitive informations
- Other attacks
  - Above mentioned attacks are just most common examples. There are many more of
    attack types

Sunday, 15 December 2024

Kali Linux Reference for Pentesters.

Pentesters.

Pentesters are ethical hackers who look for vulnerabilities in target system, and report results to company that hired them, and help to patch (fix) found vulnerabilities before criminals use these vulnerabilities earlier.

Pentesters are also called: 'Red Team Hackers'.


Operating Systems for Pentesters.

Beside Kali Linux (successor of the BackTrack Linux), there's Parrot OS, BlackArch Linux, BackBox, probably more as well. All of these are Operating Systems made for Pentesters.

But Kali Linux is industry's standard, widely used OS for Pentesters.


Directory Structure in Kali Linux:

/bin
(binaries)
This directory contains Linux binaries like the cd and ls commands.
/sbin
(system binaries)
This directory holds system binary files that serve as administrative commands (like fdisk).
/bootThis directory contains the Linux bootloader files.
/dev
(devices)
This directory contains the device configuration files (like /dev/null ).
/sysThis is similar to /dev, which contains configurations about devices and drivers.
/etc
(etcetera)
This directory contains all the administration system files (like /etc/passwd shows all the system users in Kali Linux).
/lib
(libraries)
This directory hods the shared libraries for the binaries inside /bin and /sbin.
/proc
(processes)
This directory contains the processes and kernel information files.
/lost+foundAs in the name, this directory contains the files that have been recovered.
/mnt
(mount)
This directory contains the mounted directories (example, a remote file share).
/mediaThis directory holds the removable media mounted directories (like DVD).
/opt
(option)
This directory is used for add‐on software package installation. It is also used when installing software by users (example, hacking tools that you download from GitHub).
/tmp
(temporary)
This is a temporary folder used temporarily, the holdings are wiped after each reboot. The tmp folder is a good place to download our tools for privilege escalation once we got a limited shell.
/usr
(user)
This directory contains many sub-directories. In fact, /usr/share/ is a folder that we need to memorize because most of the tools that we use in Kali Linux (like Nmap, Metasploit, etc.) are stored there, and it also contains the wordlist dictionary files (/usr/share/wordlists).
/homeThis is the home for Kali Linux users (example /home/kali/).
/rootHome directory for root user.
/srv
(serve)
This folder contains some data related to system server functionalities (like data for FTP servers).
/var
(variable)
This folder contains variable data for databases, logs, and websites. For an example, /var/www/html/ contains the files for the Apache2 web server.
/run
(runtime)
This directory holds runtime system data (like currently logged‐in users).


Commands & Important ideas.

Basic Commands:

- pwd: print working directory
- cd: change directory
- sudo: executing commands with root privileges
- ls: listing directory contents
- ls -l: as ls, but more details provided
- ls -a: as ls, but listing all files, not ignoring hidden ones
- man: displaying manual for commands
- man -f: short explaination of a command
- apropos: searching for commands using keywords
- history: displaying used commands history

Search Commands:

- find: detailed search
- locate: fast search using informations stored in database
- which : searching for executable files locations
- whereis: searching for executable files and related files locations

also, updating database for use of 'locate' command can be done using: 'sudo updatedb' command.

File & directory operations:

- touch: creating empty files
- mkdir: creating directories
- rm: removing files and/or directories
- tree: displaying directory tree structure
- cp: copying files and/or directories
- mv: moving files and/or directories
- echo: printing text
- cat: displaying file's contents
- wc -l: counting lines
- sort: sorting content

Streams:

- stdin: standard input (0)
- stdout: standard output (1)
- stderr: standard error (2)

Redirect operators:

- redirecting stdout (overwriting file): 1>
- redirecting stdout (adding to file): 1>>
- passing stdout to another (next) command (pipe): |
- redirecting stdin: <
- redirecting stderr (overwriting file): 2>
- redirecting stderr (adding to file): 2>>

Collecting System Info:

Informations about users:

- whoami: displays current user's name
- id: detailed information about current user & groups
- who: list of logged in users

Informations about system:

- hostname: name of the host
- uname (-a): (detailed) information about operating system
- env: displays all environment variables
- ps (aux): displays list of processes of all users
- top: monitoring processes in realtime

Network informations:

- ifconfig: displays network interfaces & their configurations
- ip addr: displays ip addresses
- netstat: displays network connections
- ss: socket statistics

Informations about devices:

- df -h: disk space usage, printed in 'human-readable' form
- lsblk: block devices list
- lsusb: USB devices list
- lspci: PCI devices list

Linux Text Editors:

Nano:

nano [file_name]: open/create file
Ctrl + G: help
Ctrl + X: exit
Ctrl + /n: move to line n
Ctrl + K: delete line
Ctrl + F: searching

Vim:

vim [file_name]: open file
i: insert text mode
ESC: return to command mode
dd: delete line
yy: copy line
p: paste
/text: search for text
:w: save
:q: exit
:wq or :x: save & exit
:q!: exit without saving
:set number: display lines numbers
vimtutor: interactive tutorial

Privileges:

Basic Privileges:

- r: read
- w: write
- x: execute

Categories:

- user
- group
- others

Main commands:

- chmod: change privileges
- chmod u+w: adding write privileges for file's owner (user)
- chmod g-w: remove write privileges for file's owner's group
- chmod o+x: adding execute privileges for other users
- chmod 644: privileges in octal notation
- chmod 755: standard privileges for directories

Special ('sticky') bits:

- SUID: execute with owner's privileges
- SGID: execute with group's privileges

System files:

- /etc/passwd: informations about users
- /etc/shadow: encrypted passwords
- /etc/group: informations about groups

Users management:

Managing users:

- useradd: creating user
- useradd -m: creating with home directory
- useradd -s: defining default shell
- useradd -g: setting primary group
- usermod: modifying user
- usermod -g: changing primary group
- usermod -aG: adding additional groups
- deluser: deleting user
- deluser --remove-home: removing with home directory

Managing groups:

- addgroup: creating group
- groups: displaying user's groups
- deluser user group: removing user from group

Managing sudo:

- sudo -l: checking sudo privileges
- visudo: safe editing of sudoers
- su: switching user
- passwd: changing password

Configuration files:

- /etc/sudoers: sudo configuration
- /etc/sudoers.d/: additional configuration files

Installing software:

APT (Advanced Package Tool):

- sudo apt update: refreshing packets list
- sudo apt upgrade: upgrades all packages to newest versions
- apt list --upgradable: shows list of available package upgrades
- apt list --installed: shows list of installed packages
- apt search [name]: searching for package
- apt show [name]: details about package
- sudo apt install [name]: installing package
- sudo apt remove [nazwa]: remove package

DPKG:

- dpkg -l: list of installed dpkg packages
- dpkg -l | grep [name]: search within packages

Additional tools:

- git clone [url] - cloning git repository
- pip install [name] - installing python packages

Useful options:

- --only-upgrade: upgrading only specified package
- -y: automatic install confirmation

Processes & services:

Managing processes:

- ps aux: processes list
- ps auxf: processes list displayed in form of tree
- kill [PID]: termination of a process
- kill -9 [PID]: forced termination of a process
- killall [name]: terminating all processes with given name
- jobs: listing background tasks
- fg %[numer]: moving task to foreground
- ctrl + Z: sending process into background
- &: executing process in background

Managing services:

- systemctl list-units --type=service --all: shows services list
- service --status-all: shows status of all services
- systemctl status [service]: detailed status
- systemctl start/stop/restart [service]: service management
- service [service] start/stop/restart: alternative service management
- journalctl -u [service]: service's logs

Monitoring:

- top - monitoring processes in realtime
- ps auxf | grep [name] - searching for processes


Wordlists.

Wordlists can be used, for example:
- for executing dictionary attacks that find passwords,
- for finding file names when we can't just list the containing directory's contents
  (webpage server's directories sometimes deny listing files in these directories).

One of the most popular wordlist is named 'rockyou.txt'.

In Kali Linux it's available in compressed form, in following path:

/usr/share/wordlists/rockyou.txt.gz


Sunday, 1 September 2024

Computers, Memory Pyramid & Code Size Optimization.

What is a Computer?

In Computer Sciences, Computer - by definition - is processor with memory and input/output devices. Any electronic device that has these is considered Computer. This includes Smartphones and many other tools.


Memory, Persistent or Transient.

There are two types of memory / pl: 'Są dwa rodzaje pamięci' /:
- Persistent / pl: 'Trwała' /,
- Transient / pl: 'Ulotna' /.

Persistent objects are those which continue to exist even after the program that created them has stopped running.

Transient objects cease to exist when program that created them stops.


Pyramid of Needs.

There are many types of memory, differing in price and speed of access.

Starting from the most expensive but fastest, there are:
- processor's registers,
- layers of the processor's cache (L1-L3, for example),
- RAM (Random Access Memory),
- persistent SSD/HDD storage.


Code Size optimization.

Smaller programs can be very quick in their execution.

When the whole program fits - for example - in L2 Processor's Cache, there's no need to reach RAM via BUS, so the code runs very quickly - as it's closer to the processor than RAM.


What if a Program doesn't fit in Transient Memory?

When a program needs to be executed, it needs to be loaded into the transient memory first.

However, Modern Operating Systems can send currently unused Program's parts & other Resources / for example: graphics image files, sound files and/or text files / from Transient Memory to Persistent SDD/HDD Memory and retrieve other Resources/Part(s) from Persistent Memory to Transient Memory / Usually from disk to RAM /.

/ pl: 'Współczesne Systemy Operacyjne mogą wysłać aktualnie niewykorzystywane części Oprogramowania i innych zasobów na dysk... i sprowadzić inne zasoby/części z pamięci trwałej do ulotnej, najczęściej do pamięci RAM' /.

Let's note, however, that loading/storing data in persistent memory is much slower than loading/storing from/to Transient RAM.

This is an automated operation in Modern Operating Systems, so programmers do not need to worry so much about that. Computers just slow down sometimes - and SSD/HDD becomes quite busy, when doing that.


This sometimes causes 'Flickering' / pl: 'Migotanie', 'Szamotanie' /, however. Code & Data is loaded/unloaded from/to persistent memory too slowly and can cause a Computer System to slow down or crash, as the Computing Resources run out. / Mostly CPU usage & Memory usage /.

So - in theory at least - a Computer can try to run larger programs than Computer has RAM.

Often it fails, but in theory this can work well.

Monday, 29 April 2024

Logic & Axioms.


'My Logic is based on different Axioms than Yours'.



There are many Logics, based on different Axioms.

/ EN: 'Axiom' = PL: 'Aksjomat' /.

Axioms are statements that are not proven, but assumed as true, taken on faith.

/ EN: 'assumption' = PL: 'założenie' /.

Depending on the Axioms used, Theorems can be proven or disproven, and the whole Mathematical & Logical Apparatus can be developed.


Basing on Boolean Algebra, double negation evaluates to confirmation, but in some languages - polish for example - double negation does not mean confirmation, it does mean emphasis on negation, giving negation more power.

There are rules for negating Quantifiers as well.

/ EN: 'negation' = PL: 'zaprzeczenie' /,
/ EN: 'confirmation' = PL: 'potwierdzenie' /,
/ EN: 'emphasis' = PL: 'nacisk' /,
/ EN: 'quantifier' = PL: 'kwantyfikator' /.


/ PL: 'nigdy nie zgodzę się na te warunki'. /

/ według Algebry Boole'a i Teorii Kwantyfikatorów wylicza się do:
'istnieje taki moment w czasie kiedy zgodzę się na te warunki'. /

/ a w mowie potocznej znaczy: 'nie istnieje taki moment w czasie kiedy zgodzę się na te warunki'. /


Therefore, Speech of the Art, Literature, can have the Logic based on different Axioms than Boolean Algebra.

We can say that: 'Life is more than Boolean Algebra and Quantifiers' when we want to use casual, non-logical talks.

When Boolean Algebra and Quantifiers are useful then? It's useful in computer programming, or when we want to talk logically and precisely, or when we want to express our wishes logically and precisely. But when we opt for logical and precise speech, let's make sure first that other people we talk with understand our logic.


We can also define the addition operation / it's an Axiom too / differently as well.

We can have an exception:

for example:

1+1 = 2, 1+2 = 3, 2+1 = 3, 1+3 = 4, 3+1 = 4, 2+2 = 5, 1+4 = 5, 4+1 = 5, 2+3 = 5, 3+2 = 5, 1+5 = 6, 5+1 = 6, 2+4 = 6, ...


We can also redefine the addition operation differently on the more general, more universal scale:

for example:

n+2 in our redefined addition operation is n+2+1 in classical addition operation.


By doing so, by changing Axioms, we just have revolutionized the Mathematics. ;)

Many different theorems apply now, but at least we know that we can make expression 2+2 = 5 to be evaluated as true in a certain Context - even if this brings more or less desired effects in process ;).


We are free to assume any Axioms we want, examples can be multiplied infinitely.


Which Logic is 'better' than other, then?

... it depends on the assumed Criteria, which might be Axioms as well.

Tuesday, 6 February 2024

The Enigma Cipher of WW2 & the Turing Machine.

Polish and British mathematicians were among the best of people who cracked the Adolph Hitler's cipher named Enigma, it happened during the World War 2nd.

Enigma breaking was hard, and the mathematicians were hunted by Germany's spies.

Enigma evolved, so parts of the cipher were to be cracked again and again. It was not about making an automaton once, and letting it work for the rest of WW2 ... but statistically it worked so the effort was continued.

Doing maths when time flew and lives were at stake.... so stressful. The Germany's spies added to the dangers & to the stress too.

Polish mathematicians had one of few of first computers ... it was nicknamed: 'Bomb', for it was so big invention. It increased efficiency of the enigma cipher cracking.

And there was Alan Turing's effort of course. He was a British scientist who laid foundation-theory of computer's construction.

His thinking is still present in computer sciences of modern days.

(We were taught about Languages & Automatons in Warsaw University when i was studying computer sciences. Turing's Machine was a part of this lecture).

> [ https://en.wikipedia.org/wiki/Turing_machine ].

The Turing Machine is programmed in a similar way to programming the Register Machine.

> [ https://en.wikipedia.org/wiki/Register_machine ].


--
Sources:

1. 'Cubits & Shrodinger's Cat. From Turing Machine to Quantum Computers' by John Gribbin.
(Polish Edition).


2. The internet (wikipedia & the ważniak mostly) and my own thinking.

3. My (unfinished because of health problems) education at Warsaw's University (Mathematics, Informatics & Mechanics Faculty).