 

Lecturer: Nguyễn Thị Thanh Vân – FIT - HCMUTE

 Introduction To OS and OS Security
 System Security Planning
 The Components of an OS Security Environment
 Vulnerabilities of OS
 Secure an operating system
 Operating Systems Hardening
o Linux/Unix Security
o Windows Security
 Virtualization Security

20/09/2017 2
 Operating system: collection of
programs that allows user to
operate computer hardware

3
 Three layers:
o Inner layer, computer hardware
o Middle layer, operating system
o Outer layer, different software

4
 Key functions of an operating system:
o Multitasking, multisharing
o Computer resource management
o Controls the flow of activities
o Provides a user interface
o Administers user actions and accounts
o Runs software utilities and programs
o Enforce security measures
o Schedule jobs
o Provide tools to configure the operating system and hardware

5
Why do we need to trust the operating system?
(a trusted computing base or TCB)
What requirements must it meet to be trusted?
TCB Requirements:
1. Tamper-proof,
2. Complete mediation,
and
3. Correct
 Think carefully about what you are trusting with your information
o if you type your password on a keyboard, you’re trusting:
• the keyboard manufacturer
• your computer manufacturer
• your operating system
• the password library
• the application that’s checking the password
o TCB = set of components (hardware, software, wetware) that
you trust your secrets with
 Public web kiosks should *not* be in your TCB
o should your OS?
• but what if it is promiscuous? (e.g., IE and active-X extensions)
o how about your compiler?
• A great read: “Reflections on Trusting Trust”.
TCB Controls access to protected resources

●Must establish the source of a

request for a resource
(authentication is how we do it)

●Authorization or access control

●Mechanisms that allow various

policies to be supported
 A compromised OS can
compromise a database
environment
 Physically protect the
computer running the OS
(padlocks, chain locks,
guards, cameras)
 Model:
o Bank building (operating system)
o Safe (database)
o Money (data)

9
 Used as access
points to the
database
 Three components:
o Services
o Files
o Memory

10
 Main component of operating system security
environment
 Used to gain access to the OS and its features
 Include
o User authentication
o Remote access
o Administration tasks
o Password policies

11
 Common threats:
o File permission
o File sharing
 Files must be protected from unauthorized reading and
writing actions
 Data resides in files; protecting files; protects data

12
 Read, write, and execute privileges
 In Windows:
o Change permission on the Security tab on a file’s Properties
dialog box
o Allow indicates grant; Deny indicates revoke
 In UNIX/Linux
o Three permission settings: owner; group to which owner
belongs; all other users
o Each setting consist of rwx
• r for reading, w for writing, and x for executing
o CHMOD command used to change file permissions

13
$ chmod 644 mail_list

14
 FTP (File Transfer Protocol):
o Internet service for transferring files from one computer to another
o Transmits usernames and passwords in plaintext
o Root account cannot be used with FTP
o Anonymous FTP: ability to log on to the FTP server without being
authenticated
 Best practices:
o Use Secure FTP utility if possible
o Make two FTP directories:
• One for uploads with write permissions only
• One for downloads with read permissions only
o Use specific accounts with limited permissions
o Log and scan FTP activities
o Allow only authorized operations

15
 Naturally leads to security risks and threats
 Peer-to-peer programs: allow users to share
files over the Internet
 Reasons for blocking file sharing:
o Malicious code
o Adware and spyware
o Privacy and confidentiality
o Pornography
o Copyright issues

16
 Hardware memory available on the system can
be corrupted by badly written software
 Can harm data integrity
 Two options:
o Stop using the program
o Apply a patch (service pack) to fix it

17
20/09/2017 18
 Authentication: Verifies user access to the operating system
 Physical authentication:
o Allows physical entrance to company property
o Magnetic cards and biometric measures
 Digital authentication: verifies user identity by digital means
 Digital certificates: identifies and verifies holder of certificate
 Digital token (security token):
o Small electronic device
o Displays a number unique to the token holder;
o Uses a different password each time
 Digital card: Also known as a security card or smart card
o Similar to a credit card; uses an electronic circuit instead of a magnetic strip
o Stores user identification information
 Kerberos:
o Developed by MIT
o Uses tickets for authentication purposes
19
 Lightweight Directory Access Protocol (LDAP):
o Developed by the University of Michigan
o A centralized directory database stores:
• Users (user name and user ID)
• Passwords
• Internal telephone directory
• Security keys
o Efficient for reading but not suited for frequently changing information
 NT LAN Manager (NTLM):
o Developed and used by Microsoft
o Employs a challenge/response authentication protocol
 Public Key Infrastructures (PKI):
o User keeps a private key
o Authentication firm holds a public key
o Encrypt and decrypt data using both keys

20
 RADIUS: used by network devices to provide a
centralized authentication mechanism
 Secure Socket Layer (SSL): authentication
information is transmitted over the network in
an encrypted form
 Secure Remote Password (SRP):
o Password is not stored locally
o Invulnerable to brute force or dictionary attacks

21
 Process that decides whether users are permitted to
perform the functions they request
 Authorization is not performed until the user is
authenticated
 Deals with privileges and rights

22
 Create user accounts
 Set password policies
 Grant privileges to users
 Best practices:
o Use a consistent naming convention
o Always provide a password to an account and force the user to
change it at the first logon
o Protect passwords
o Do not use default passwords

23
 Best practices (continued):
o Create a specific file system for users
o Educate users on how to select a password
o Lock non-used accounts
o Grant privileges on a per host basis
o Do not grant privileges to all machines
o Use ssh, scp, and Secure FTP
o Isolate a system after a compromise
o Perform random auditing procedures

24
 First line of defense
 Dictionary attack: permutation of words in
dictionary
 Make hard for hackers entering your systems
 Best password policy:
o Matches your company missions
o Enforced at all level of the organization

25
 Best practices:
o Password aging
o Password reuse
o Password history
o Password encryption
o Password storage and protection
o Password complexity
o Logon retries
o Single sign-on enables a user to log in once and gain access to
the resources of multiple software systems without being
prompted to log in again

26
 Tool must widely used by public
 May be the tool must frequently used by hackers:
o Viruses; Worms; Spam; Others
 Used to send private and confidential data as well as offensive
material
 Used by employees to communicate with:
o Clients
o Colleagues
o Friends
 Recommendations:
o Do not configure e-mail server on the same machine where sensitive
data resides
o Do not disclose technical details about the e-mail server

27
 Top vulnerabilities to Windows systems:
o Internet Information Services (IIS)
o Microsoft SQL Server (MSSQL)
o Windows Authentication
o Internet Explorer (IE)
o Windows Remote Access Services National Vulnerability
o Microsoft Data Access Components (MDAC) Database:
o Windows Scripting Host (WSH) http://nvd.nist.gov/
o Microsoft Outlook and Outlook Express
o Windows Peer-to-Peer File Sharing (P2P)
o Simple Network Management Protocol (SNMP)

28
 Top vulnerabilities to UNIX systems:
o BIND Domain Name System National Vulnerability
o Remote Procedure Calls (RPC)
Database:
o Apache Web Server http://nvd.nist.gov/
o General UNIX authentication accounts with no passwords or
weak passwords
o Clear text services
o Sendmail
o Simple Network Management Protocol (SNMP)
o Secure Shell (SSH)
o Misconfiguration of Enterprise Services NIS/NFS
o Open Secure Sockets Layer (SSL)

29
 Basic steps
o Install and patch the operating system.
o Harden and configure the OS to adequately address the
identified security needs of the system by:
• Removing unnecessary services, applications, and protocols.
• Configuring users, groups, and permissions.
• Configuring resource controls.
o Install and configure additional security controls, such as anti-
virus, hostbased firewalls, and IDS, if needed.
o Test the security of the basic OS to ensure that the steps taken
adequately address its security needs.

20/09/2017 30
 Hardening:
o attempting to make OS bulletproof.
o Ideally - leave OS exposed to the general public on the Internet
without any other form of protection.
o A hardened system should serve only one purpose--it's a Web
server or DNS or Exchange server, and nothing else. These
systems need too many functions to be properly hardened.

20/09/2017 31
 Disable all unnecessary services.
o determine which services can be disabled.
• Remote Procedure Call (RPC) service.
• little documentation exists to identify what services a given purpose will
require.
• knowing which services are required and which can be disabled is
largely a matter of trial and error.
 Remove all unnecessary executables and registry
entries.
o Forgetting to remove unneeded executables and registry entries
might allow an attacker to invoke something that had previously
been disabled.
 Apply appropriately restrictive permissions to files,
services, end points and registry entries.
o Inappropriate permissions could give an attacker an opening.
o The ability to launch CMD.EXE as "LocalSystem," for example, is a
classic backdoor.

20/09/2017 32
 Adjusting retransmission of SYN-ACKS. This makes
connection responses time out more quickly during a SYN flood .
 Determining how many times TCP retransmits an
unacknowledged data segment on an existing connection. TCP
retransmits data segments until they are acknowledged or until this
value expires.
 Disabling ICMP Router Discovery Protocol (IRDP) where an
attacker may remotely add default route entries on a remote system .
 Disabling these services: Telnet, Universal Plug and Play Device
Host, IIS, Disable Guest accounts
 Use the Local Security Policy
 Disable File and Print Sharing.
 Disable Remote Assistance and Remote Desktop
 Use NTFS File system.
 Disable auto-logins.

20/09/2017 33
 Encrypt Data Communication
 Avoid Using FTP, Telnet, And Rlogin / Rsh Services
 Minimize Software to Minimize Vulnerability
 One Network Service Per System or VM Instance
 Keep Linux Kernel and Software Up to Date
 Use Linux Security Extensions
 SELinux
 Password: Policy, Aging, Empty
 Login:
o Locking User Accounts After Login Failures
o Make Sure No Non-Root Accounts Have UID Set To 0
o Disable root Login

20/09/2017 34
 Disable Unwanted Services
 Find Listening Network Ports
 Configure Iptables and TCPWrappers
 Linux Kernel /etc/sysctl.conf Hardening
 Separate Disk Partitions
 Disk Quotas
 Turn Off IPv6
 Disable Unwanted SUID and SGID Binaries
 Logging and Auditing
 Secure OpenSSH Server
 Install And Use Intrusion Detection System
 Disable USB/firewire/thunderbolt devices

20/09/2017 35
• A VM is a software implementation of a machine that execute programs like a
physical machine
• A VM can support individual processes or a complete system depending on
the abstraction level where virtualization occurs.
• Virtualization – a technology that allows running two or more OS side by side
on one PC or embedded controller
•q
 More security implications because of the VM is installed that communicates
reliance on the underlying OS, used in directly with system hardware rather
VMware and MS Virtual PC than relying on a host OS

Hosted Bare - Metal

• Thin Virtualization – reduced size, independence
=> Get Strong Security in a Small Package
• the attack surface much smaller, and reduces the potential for vulnerabilities
• far fewer interfaces to exploit and less malware threats
• Security Concepts in Architecture
 Extended computing stack
 Guest isolation
 Host Visibility from the Guest
 Virtualized interfaces
 Management interfaces
 Greater co-location of data and assets on one box
 Abstraction and Isolation
 Better Forensics and Faster Recovery After an Attack
 Patching is Safer and More Effective
 More Cost Effective Security Devices
 Future: Leveraging Virtualization to Provide Better
Security
 VM Sprawl
 Mobility
 Hypervisor Intrusion
 Hypervisor Modification
 Communication
 Denial of Service
Issue Hosted Bare-Metal
susceptible to all the
Vulnerability of the
vulnerabilities and attacks that a much smaller attack surface
underlying OS
are prevalent on such systems.

Sharing of files
there is no mechanism share user
and data between vulnerable to data leakage and
information between virtual machines
the guest and the malicious code intrusion.
and their host.
host

Resource They are at the mercy of the host No single virtual machine can use all
allocation OS and other applications. the resources or crash the system.

- is targeted for environments can potentially be exposed to

where the guest virtual machines malicious users and network traffic.
can be trusted. (software Strong isolation and strict separation
Target Usage
development, testing, of management greatly reduce any
demonstration, and trouble- risk of harmful activity going beyond
shooting.) the boundaries of the virtual machine.
 Managing oversight and responsibility
 Patching and maintenance
 Visibility and compliance
 VM sprawl
 Managing Virtual Appliances
 Introduction To OS and OS Security
 System Security Planning
 The Components of an OS Security Environment
 Vulnerabilities of OS
 Secure an operating system
 Operating Systems Hardening
o Linux/Unix Security
o Windows Security
 Virtualization Security

21/09/2017 45
20/09/2017 46