The official documentation is extremely helpful and is a must read before starting anything. The ELK stack is proving to be a very popular suite of tools, and good documentation abounds on the internet.

This article presents some configuration scripts and research links that were used to provision the system and some tips and tricks learned during implementation and R&D of the system This seemed right up elasticsearch’s alley, and the more we use the system, the more adept at this sort of use case it turns out to be. The amount of data generated is quite large, ranging into terabytes of logs and events.

The platform would then be used for queries during forensic investigations and to help follow up on Indicators of Compromise. On the Linux setup, I have also tried setting a filename instead of '*.raw' but that did not work either.During a recent project we were required to build a “Logging Forensics Platform”, which is in essence a logging platform that can consume data from a variety of sources such as windows event logs, syslog, flat files and databases. I used this same syntax on a Windows setup to test it, which worked it successfully removed files. In the debug log, I do not see an attempt to try and match files in the directory for removing: #Įxec file_remove('/eventarchive/processed/*.raw', (now())) As seen, I used a time interval of every 1 minute as I do not know the best way to have the file deleted after processing. Here is my config that does not throw any syntax errors when starting nxlog. From the nxlog documentation, I am using file_remove, which is not working. I am currently trying to remove a file after nxlog has finished processing. I am deploying nxlog on a Linux server (Red Hat Enterprise release 6.6).

Nxlog flat file