EventLog Analyzer's

log storage mechanism

www.eventloganalyzer.com
Table of contents

Overview 1

Metadata of logs 1

Indexes of the log data 1

Log archival 2

Securing log data 3

Fault tolerance mechanism 3

www.eventloganalyzer.com
 1. Overview
EventLog Analyzer is a comprehensive log management, auditing, and IT compliance
management solution that collects, analyzes, correlates, and stores logs.

EventLog Analyzer stores log data in three different formats:

Metadata of logs
Indexes of the log data
Log archival

2. Metadata of logs
Metadata information such as log count, time frame, and more, that are collected from devices
across the network are stored in the solution’s database. By default, EventLog Analyzer uses
PostgreSQL as its backend database. However, you can migrate to MS SQL database if you wish
to. The table below shows the default retention period. Please note that the retention period can
be customized by navigating to Settings > Admin settings > DB Retention Settings.

DESCRIPTION DEFAULT

Log Retention 32 days

Table 1: Log retention

3. Indexes of the log data

When you perform a log search, the solution refers to this index to retrieve the required data.
These indexes are stored in Elasticsearch (ES) directory. The default storage location of
Elasticsearch Directory is EventLog Analyzer/ES. This location can be customized. To avoid latency,
it is not recommended to use NAS Storage for ES.

The size of the Elasticsearch Directory is directly proportional to the log flow and the ES retention
period. The table below shows the default retention period of the indexed logs. Please note that
the retention period can be customized by navigating to Settings > Admin settings > DB Retention
Settings. There is also an option to store the data present in ES folder as zip files to save space.
When you search for log data that's zipped, the files will automatically be unzipped and processed.

1 www.eventloganalyzer.com
 DESCRIPTION DEFAULT

ES - Retention 32 days

ES - Zip logs 32 days

Table 2: ES retention period

4. Log archival
To meet compliance regulatory mandates, you need to archive your log data. EventLog Analyzer
helps archiving log data periodically. The collected log data is archived in EventLog
Analyzer\archive directory. You can change this location as well. By default, the logs are retained
forever. However, you can customize this settings and choose a retention period based on your
requirements. Please note that the size of log archival is directly proportional to your log flow. So
make sure that you don’t consume much space and move your old archival to external or NAS
storage.

Below table elaborates the archival mechanism of the product. The log files will be deleted once
the retention period gets over.

DESCRIPTION DEFAULT

Archive File's Forever

Retention Interval

Flat File Rotation Once in 12 hours (or)

Interval Size of each flat file
reaches 250 MB

Zip Creation Interval Once in 4 days (or) Number

of flat files> 8 files

Table 3: Archive settings

1. Flat file management

The log data collected by EventLog Analyzer is written in a flat file.For every 12 hours, logs will get
archived to a new flat file. This time interval can be customized. The flat file created is categorized
based on the log format related to the device from which the logs are being collected. Further, the
flat files will be split and retained if the configured time interval or storage threshold (250 MB) limit
is reached. You can customize the default time interval and storage threshold limit.

2 www.eventloganalyzer.com
 ZIP management
The flat file generated by the solution are compresses in 20:1 ratio and stored as ZIP files for
effective space utilization. By default, a zip file will be created for every 4 days or if the flat file
present in the directory reaches the threshold limit of eight files. These default time interval and
storage threshold limit can be customized.

5. Securing log data

EventLog Analyzer uses dynamic encryption and time stamping techniques to secure the log
data's integrity. It also generates alert (via Email) when data tampering incidents occur. You can
also employ Base64 encryption algorithm to secure log data. By default, the solution disables
encryption of log data. You can enable it by navigating to Settings > Admin settings > Manage
Archives > Settings > Encrypt Data.

6. Fault tolerance mechanism in EventLog Analyzer

When the storage medium used by EventLog Analyzer is either down or having issues, the fault
tolerance mechanism comes into picture, which enables the solution to continue its intended
operation, possibly at a reduced level rather than failing completely.

This section briefs on how the solution handles storage medium failures.

1. If EventLog Analyzer's ES is a shared folder and is unavailable, then the product

will not be able to redirect the logs to different path for failover. Hence, Hence, we
recommend you to use the failover or disaster functionality available in the storage
devices that you use.

2. EventLog Analyzer will be unable to redirect the archives to a different path for
failover due to the product's architectural complexity. Hence we recommend using
the basic failover or disaster functionality available in the storage devices that you use.

3 www.eventloganalyzer.com
General recommendation to optimize disk space
You can optimize the disk space based on the volume of log data that you collect and/or the
retention period until which the log data is stored.

Log volume-based optimization: The hard disk space required to store the log data
is directly proportional to the volume of log data generated in your environment.
For a high log flow rate, you need to allocate a larger disk space to store and process
the log data. However, if the need for disk space is growing alarmingly, then you can
optimize your log gathering by collecting only the logs that are required.
We recommend you to:

Disable auditing functionality for irrelevant Windows events

Ensure that only the necessary syslogs are being forwarded for monitoring

You can employ log collection filters to remove the noise and reduce
the disk space utilization.

Retention-based optimization: The archived logs and index folders majorly contribute
to the growing size of disk space. The total disk space required at any time to store the
logs generated by your network is a combination of the size of archive and index folders.
To optimize the disk space, you need to:

Minimize the archive retention period

Minimize the database retention period to reduce the indexed data volume.

EventLog Analyzer is a web-based, real-time log management and IT compliance solution that combats
network security attacks. With comprehensive log management capabilities, EventLog Analyzer helps
organizations meet their diverse auditing needs. It also offers out-of-the-box compliance reports and alerts
that meet stringent IT regulatory mandate requirements with ease.

For more information about EventLog Analyzer, visit manageengine.com/eventloganalyzer.