Audit Trails

An audit trail is a series of records of computer events, about an operating system, an application, or user activities. It is generated by an auditing system that monitors system activity. Audit trails have many uses in the realm of computer security :

Individual Accountability : An individual's actions are tracked in an audit trail allowing users to be personally accountable for their actions. This deters the users from circumventing security policies. Even if they do, they can be held accountable.

Reconstructing Events : Audit trails can also be used to reconstruct events after a problem has occurred. The amount of damage that occurred with an incident can be assessed by reviewing audit trails of system activity to pinpoint how, when, and why the incident occurred.

Problem Monitoring : Audit trails may also be used as on-line tools to help monitor problems as they occur. Such real time monitoring helps in detection of problems like disk failures, over utilization of system resources or network outages.

Intrusion Detection : Intrusion detection refers to the process of identifying attempts to penetrate a system and gain unauthorized access. Audit trails can help in intrusion detection if they record appropriate events. Determining what events to audit so that audit trails can be used in an effective manner to aid intrusion detection is one of the present research issues being looked into by the research community.

Model of Auditing System

A simple model of an auditing system consists of two parts :

Audit Data Collector : This is responsible for collecting the audit data
Audit Data Analyzer : This is responsible for analyzing the audit data transferred to it by the Audit Data Collector

Often a file is used to transfer the audit data from the collector to the analyzer. This leads to problems when the audit data is collected from different systems. This is due to the lack of a standard interface for audit trails. Developing standards for the format and content of audit trails is an ongoing research effort.

Audit Trail Standards

Development of audit trail standards for audit format and content that support security goals and that gain wide acceptance is an important step in overcoming incompatibility issues.

Format Standards

A standard format with wide acceptance would help overcome incompatibility and interoperability issues faced by the developers of audit data analysis systems. It would also allow the exchange of audit data from audit sources on different systems and facilitate collaborative analysis of data in a networked environment.

Some of the proposed standards for the format of audit trails are :

1. Bishop's Standard Audit Trail Format

Bishop [Bis95] states that a standard format must be both extensible and portable to meet the needs of different heterogeneous systems and transportability across various systems and network protocols. Bishop defines a standard log record format that is both portable and extensible. Each log record consists of fields separated by a field separator ('#') and delimited by start and stop symbols ('S' and 'E'). The number of fields is not fixed to meet the need for extensibility. All values are ASCII strings. This avoids the issues of byte ordering and floating point format. This format however does not attempt to standardize the fields of an audit trail record.

2. Normalized Audit Data Format (NADF)

The Normalized Audit Data Format (NADF) [Mou93, Mou97] was defined by the developers of the ASAX misuse detection system to provide a degree of operating system independence. A NADF audit trail is a sequential file of NADF records. Any audit trail can be converted to the NADF format. During conversion, the audit records of the native audit trail are abstracted into a sequence of audit data values. Each audit data value is stored in a separate NADF record consisting of three fields:

          Identifier : the type of the audit data value.
          Length     : the length of the audit data value.
          Value       : the audit data value.

3. svr4++ Common Audit Trail Interchange Format for UNIX

This is a standard meant specifically for the Unix systems. The attributes entered in a audit record are time, event type, process identifier, result, user and group information, session identifier, labeling information for the process, information about the object and miscellaneous data. The attributes are all in ASCII. This standard approaches portability but lacks the features for extensibility.

Content Standards

The content of the audit trails also needs to be standardized. This would help analyzing audit data from different audit sources and improve interoperability in a networked environment. Some of the proposed standards are :

1. DoD Trusted Computer System Evaluation Criteria

This is a standard created by the National Computer Security Center against which a computing system can be evaluated for security assurance. There are four classes of criteria namely A, B, C, and D, with systems meeting the criteria of the highest division (A) providing the best level of security assurance. There are a number of subdivisions in classes B and C. Classes C2 through A1 require the ability to audit security relevant activities on the system. This standard specifies what events are to be audited and what each event should contain.

For example, in a C2 system, events that must be audited include use of identification and authentication mechanisms, introduction and deletion of objects, administrative actions, and other security relevant events. Moreover, each audited event should contain the following information : date and time of the event,user identifier, type of event, success or failure of the event, origin of request for identification/authentication events and name of object for object introduction/deletion events.

2. Security Criteria for Distributed Systems

This is a standard created by the Institute for Defense Analyses in 1995 for distributed systems. This standard specifies various types of events to be audited. The events are grouped into six categories:

   1. Access Control and Administrative Policy Events
   2. Data Confidentiality and Integrity Policy Events
   3. Non-Discretionary Policy Events
   4. Availability Policy Events
   5. Cryptographic Policy Events
   6. Default and Dependent Events

The standard specifies the information to be recorded in each event to be: date and time, subject attribute information, identity of host generating the audit record, event class and event identifier within the class, and event outcome (success or failure).

Distributed Auditing

Distributed Auditing allows the audit data to be collected in different systems in the network. This is necessary to provide security in a networked environment because the correlation of user activities taking place at different hosts in the same network might reveal a malicious behavior while the same behavior might seem legitimate at a single host level. [SM91] discusses a lot of issues that affect auditing in a distributed environment. These include collection and storage, protection, integration and analysis.

In [BEF+91], a Distributed Auditing System (DAS) architecture for the distribution and collection of audit data in a distributed environment is presented. It addresses issues of migrating audit data from a collection point to an analysis point, and management of audit functions from a remote location.

Audit Trail Analysis

The audit trails need to be analyzed to determine vulnerabilities, establish accountability, assess damage and recover the system. Manual analysis of audit trails though cumbersome is often resorted to because of the difficulty to construct queries to extract complex information from the audit logs. There are many tools that help in browsing the audits. The major obstacle in developing effective audit analysis tools is the copious amounts of data that logging mechanisms generate.

ComputerWatch Audit Trail Analysis Tool

Automated Audit Analysis

There has been a lot of work done in the area of automated audit analysis, mainly for intrusion detection purposes. These tools use the audit data as input. These tools are based on three approaches namely :

Statistical

Automated statistical systems such as SRIs IDES [Denn87, Javi91] and Haystack Laboratory's Haystack [Sma88] focus primarily on defining characteristics of a normal user or group, which generally involves a period of training; then they employ statistical measures to determine if a current users characteristics match his previously observed behavior. This approach is also called "Anomaly Detection."

Rule-based expert systems

Automated expert systems such as portions of IDES [Lunt89, Garv91], DIDS [Snap91a], Wisdom & Sense (W&S)[Vacc89], and signature analysis [Snap91b] pursue a different approach. Instead of detecting anomalies, these systems attempt "misuse detection" by using a priori rules that are indicative to a human expert of an intrusion.

Machine learning

Application of machine learning to intrusion detection problem is a relatively new approach. Machine learning attempts to monitor and learn the normal activities of users. By knowing past events, inductive learning algorithms try to predict later events.

Distributed Audit Analysis

Distributed Audit Analysis is needed for network security because as said before, the correlation of users actions taking place at different hosts could reveal a malicious behavior while the same actions may seem legitimate at a single host level. This has the following benefits over a centralized audit trail analysis :

1. It drastically reduces the network traffic when compared to the centralized analysis where all audit data are sent to a central host for analysis.
2. It also achieves a balance of the CPU time over several machines as the analysis is being done on several machines instead of overloading the central host as in centralized audit analysis.

[MCZH95] discusses in detail about the architecture for distributed audit trail analysis and its benefits over single audit trail analysis.

Research Issues

The various issues in audit trails that are being looked into by the research community are listed below :

Audit Content

This area of research aims at determining the format and content of the audit data that is needed for detecting computer intrusions and misuse. The Audit Trails Format Group at CERIAS, Purdue is pursuing this research issue.

Audit Analysis

This area of research deals with issues like automated analysis, distributed analysis, more efficient and effective ways of audit analysis for intrusion detection. Most of the research groups working on intrusion detection are involved in this.

Audit Compression

This area of research aims to develop techniques and, ultimately, tools to efficiently reduce audit data, both in the sense of economizing storage space and in the sense of abstracting higher-level, more useful information for security administrators. The Audit Trail Reduction Group at CERIAS, Purdue is pursuing this research issue.

Audit Tamperproofing

This area of research aims at securing the audit logs from tampering. [SK99] deals with the issue of securing audit logs to support computer forensics. The other papers related to this issue are [SK98] and [SK99(2)] .

Commercial OS Audit Trail Formats

The audit trail formats in commercial operating systems like those listed below can be obtained from operating system administration and reference manuals. They are well summarized in [KP97]. The main features are listed below :

Solaris

The Solaris operating system from Sun Microsystems includes a security extension called the Basic Security Module or BSM. This BSM provides enhanced security auditing that is designed to achieve the C2 level in the Trusted Computer System Evaluation Criteria [Nat85].

HP-UX

The HP-UX trusted operating system [Hew95, Hew96] from Hewlett Packard provides auditing capability. The auditing system records occurrences of access by subjects to objects for detection of attempts to bypass protection mechanisms or to misuse privileges. Audit records are generated both by the system as well as by self-auditing applications.

OpenVMS VAX

The OpenVMS VAX operating system [Dig96a, Dig96b] from Digital Equipment Corporation provides an auditing system that supports monitoring of security relevant activities. It can record both successful and unsuccessful security relevant activity and allows applications to contribute security relevant event information to audit log.

Windows NT

The Windows NT operating system [Mic95b, Mic95a] from Microsoft Corporation includes mechanisms for recording significant events related to both the operating system and the applications. Windows NT is designed for C2-level security of the Trusted Computer System Evaluation Criteria [Nat85]. Each application is allowed to define and log its own auditable events.

Books

Audit and Control of Computerized Systems Jauier F. Kuong / Paperback / Published 1983

Audit and Control of Data Communication Networks (Computer Security Auditing and Controls Ser.) Chester M. Winters / Hardcover / Published 1987

Audit Trail Administration, Unix Svr 4.2 Unix Systems Lab / Paperback / Published 1993

Client/ Server Control, Security And Audit (MAP-34) Javier F. and MASP Consulting Staff Kuong / Hardcover / Published 1996

Edi : Audit and Control I. Walden, A. Braganza / Hardcover / Published 1993

Edi Security, Control, and Audit Albert J., Jr. Marcella, et al Published 1993

How to Train, Develop and Manage the Edp Audit and Control Resource J. F. Kuong / Paperback / Published 1982

Microcomputers : Security, Auditability & Controls J.F. Kuong, et al / Hardcover / Published 1985

Microsoft Windows Nt 4.0 Security Handbook : Guidelines for Security, Audit and Control Coopers and Lybrand / Paperback / Published 1998

Proceedings of Compsec International 1994 : Proceedings of the 11th World Conference on Computer Security, Audit and Control, London Uk, October 12-14 Paperback / Published 1994

The Security, Audit and Control of Databases Rodney Clark, et al / Hardcover / Published 1991

Papers

[BEF+91] D. Banning, G. Ellingwood, C. Franklin, C. Muckenhirn, and D. Price. Auditing of Distributed Systems. In Proceedings of the 14th National Computer Security Conference, pages 59-68, October 1991.
This paper proposes the Distributed Auditing System (DAS) architecture and addresses many issues related to auditing in distributed systems.
[Bis95] M. Bishop , "A Standard Audit Log Format". Proc. of the 1995 National Information Systems Security Conference.Baltimore, Maryland, October 10-13, 1995, pp. 136-145.
This paper identifies the need to standardize the format of audit trails and proposes a standard.
[Bis89] M. Bishop, "A Model of Security Monitoring," Proceedings of the Fifth Annual Computer Security Applications Conference, 1989, pp. 46-52
[Denn87] Dorothy E. Denning, David Edwards, R. Jagannathan,Teresa Lunt, and Peter G. Neumann, "A Prototype IDES - A Real-Time Intrusion          Detection Expert System," Technical Report, SRI International, August 1987.
[Dig96a]   Digital Equipment Corporation, Maynard, Massachusetts. OpenVMS Guide to System Security, November 1996. OpenVMS VAX Version 7.1.
[Dig96b] Digital Equipment Corporation, Maynard, Massachusetts. OpenVMS System Management Utilities Reference Manual, November 1996         Open-VMS VAX Version 7.1.
The above two documents can be referred to learn about the auditing features in Open-VMS VAX systems.
[Garv91] Thomas D. Garvey and Teresa Lunt, "Model-based Intrusion Detection," Proceedings of the 14th National Computer Security Conference, Washington DC, October 1991.
[Hew95] Hewlett-Packard Company, Palo Alto, California. HP-UX System Administration Tasks, second edition, June 1995. B2355-90079.
[Hew96]   Hewlett-Packard Company, Palo Alto, California. HP-UX Reference Manual Volume 4: Sections 4, 5, and 7, third edition, July 1996. B2355-90120.
The above two documents can be referred to learn about the auditing features in HP-UX systems.
[HK88]   Lawrence R. Halme and Brial L. Kahn. Building a Security Monitor with Adaptive User Work Pro les. In Proceedings of the 11th National Computer Security Conference, pages 274-283, October 1988.
[Javi91]   Harold S. Javitz and Al Valdez, "The SRI IDES Statistical Anomaly Detector," Proceedings of the IEEE Symposium on Security and Privacy, Oakland,CA, May 1991.
[JDS91]   Kathleen A. Jackson, David H. DuBois, and Cathy A. Stallings. An Expert System Application for Network Intrusion Detection. In Proceedings of the 14th National Computer Security Conference, pages 215-225,October 1991.
[KP97]   Host-Based Misuse Detection and Conventional Operating Systems' Audit Data Collection,Kathrine Price,Department of Computer Sciences, COAST TR 97-15
This document is a comprehensive work which describes the basics of audit trails, the audit formats in commercial operating systems, the requirements of popular intrusion detection systems and tries to show that the existing auditing features don't meet the expectations of the intrusion detection systems.
[Lunt89]   Teresa Lunt, R. Jagannathan, Rosanna Lee, Alan Whitehurst, and Sherry Listgarten, "Knowledge-Based Intrusion Detection", Proceedings of the AI Systems in Government Conference,Washington DC, March 1989.
[Nat85] National Computer Security Center. Department of Defense Trusted Computer System Evaluation Criteria, December 1985. DoD 5200.28-STD.
This document describes the DoD Trusted Computer System Evaluation Criteria (TCSEC) against which a computing system can be evaluated for security assurance.
[Mou93] N. Habra, B. Le Charlier, A. Mounji,Advanced Security Audit Trail Analysis on uniX. Implementation Design of the NADF Evaluator.       Research Report, March 1993.
[Mou97] Abdelaziz Mounji. Languages and Tools for Rule-Based Distributed Intrusion Detection. PhD thesis, Institut d'Informatique, Facultes          Universitaires Notre-Dame de la Paix, Namur, Belgium, September 1997.
The above two documents address the issue of the standardizing audit formats and propose the NADF format.
[Mic95a] Microsoft Corporation. Microsoft Win32 Programmer's Reference, 1995.
[Mic95b] Microsoft Corporation. Windows NT Resource Guide, 1995.
The above two documents describe the auditing features of Windows NT
[MHL94] Biswanath Mukherjee, L. Todd Heberlein, and Karl N. Levitt. Network Intrusion Detection. IEEE Network, 8(3):26-41, May/June 1994.
[MCZH95] Abdelaziz Mounji, Baudouin Le Charlier, Denis Zampunieris, and NajiHabra. Distributed Audit Trail Analysis. In Proceedings of the the    ISOC 1995 Symposium On Network and Distributed System Security, pages102-112, 1995.
This paper discusses the issue of distributed analysis of audit trails and compares single audit trail analysis features with the distributed audit trail analysis. It also proposes an architecture for the same and looks into the pros and cons of it.
[Pic87] J. Picciotto. The Design of An Effective Auditing Subsystem. In Proceedings of the IEEE Symposium on Security and Privacy, pages 13-22, April 1987.
This paper describes the design and implementation of the Compartmented Mode Workstation 's(CMW) auditing subsystem. It gives a broad picture of the auditing subsystem. It also gives details on the events that are audited in CMW system.
[RS91]   Deborah Russel and G.T. Gangemi Sr. Computer Security Basics. O'Reilly & Associates, Inc., 1991.
[SK98]   B. Schneier and J. Kelsey, "Cryptographic Support for Secure Logs on Untrusted Machines," The Seventh USENIX Security Symposium Proceedings, USENIX Press, January 1998, pp. 53-62
This paper describes a computationally cheap method for making all log entries generated prior to the logging machine's compromise impossible for the attacker to read, and also impossible to undetectably modify or destroy.
[SK99] B. Schneier and J. Kelsey, Secure Audit Logs to Support Computer Forensics, ACM Transactions on Information and System Security, v.2,n.2, May 1999, pp. 159-176.
This paper describes the problem of audit logs being tampered with and suggests a scheme to secure it. It is one of the few papers which deals with audit trail tamperproofing and hence is a stating point for this issue.
[SK99(2)]   J. Kelsey and B. Schneier, Minimizing Bandwidth for Remote Access to Cryptographically Protected Audit Logs, Second International       Workshop on the Recent Advances in Intrusion Detection (RAID '99), September 1999
This paper also deals with secure audit logs. It describes how the amount of information exchange required to verify the entries in the audit log is greatly reduced. It describes how by making audit-log verification more efficient, the system is made more suitable for implementation in low-bandwidth environments.
[SM91]    Samuel I. Schaen and Brian W. McKenney. Network Auditing: Issues and Recommendations. In Proceedings of the 7th Annual Computer Security Applications Conference, pages 66-78, December 1991.
This paper describes the need for network auditing and looks at the issues involved in network auditing
[Sma88]   Stephen E. Smaha. Haystack: An Intrusion Detection System. In Proceedings of the Fourth Aerospace Computer Security Applications           Conference, pages 37{44, December 1988.
[Snap91a]   Steven R. Snapp, James Brentano, Gihan Dias,Terrance Goan, Louis Todd Heberlein, Che-lin Ho,Karl Levitt, Biswanath Mukherjee, Stephen Smaha,Tim Grance, Daniel Teal, and Douglas Mansur, "DIDS(Distributed Intrusion Detection System) -Motivation, Architecture, and An Early Prototype",Proceedings of the 14th National Computer Security Conference, Washington DC, October 1991.
[Snap91b]    Steven Ray Snapp, "Signature Analysis and Communication Issues in a Distributed Intrusion Detection System," Masters Thesis, Department of Computer Science, University of California, Davis CA 95616, 1991.
[Vacc89]     H.S. Vaccaro and G.E. Liepins, "Detection of Anomalous Computer Session Activity," Proceedings of the IEEE Symposium on Security and Privacy, May1989.
[BW93]     B. Wetmore , "Audit Browsing", Masters Thesis, 1993
This document looks at the problems in audit analysis, proposed the creation of an audit workbench and looks at the issues of audit browsing. It is a good start to look at issues plaguing audit analysis with respect to intrusion detection.

Articles

A Note on Detecting Tampering with Audit Trails by Dr. Frederick B. Cohen

Genetic Algorithms, an Alternative Tool for Security Audit Trails Analysis

Electronic Commerce with Verifiable Audit Trails

Examples of Audit Trails under Unix

Reports and Guides

ITL BULLETIN - MARCH 1997

Mailing Lists

IDS Mailing List

Terminology

Audit

(1) To examine a system for security problems and vulnerabilities [RS91]
(2) To record and analyze system activity for security problems and vulnerabilities [RS91]
(3) To analyze system activity for security problems [Bis89]

Audit Trail

A chronological set of records of system activity [RS91]
The terms audit log,activity log and system log are synonymous to audit trail.

Intrusion

(1) Inappropriate use of a computer system [Sma88]
(2) Penetration of a computer system by an outsider [HK88,JDS91]

Intrusion Detection

Identifying the entity who is using or attempting to use the computer system without authorization
or who have legitimate access but are attempting to abuse their privileges [MHL94]

Page created by Rajeev Gopalakrishna in April 2000.