Compress & Delete old logs

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Compress & Delete old logs

Michael Schall
Is there a way with the file target to have it compress itself as it
is rolling to the next file?  I would name my logs by date, and would
like the old files to compress (zip, cab, some format readable without
extra installs) as a new file is being created.

Also, is it possible to have a max number of files saved?  We would
like to only keep the last x days of logs.

So day to day the files would be compressed and after x days, the
compressed files would be removed.

Is this possible currently, or planned in the future?

Mike


-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
Nlog-list mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/nlog-list
Reply | Threaded
Open this post in threaded view
|

Re: Compress & Delete old logs

Jarek Kowalski
Administrator
Well, the answer is "kind of",

1) Latest snapshots support auto-archiving feature, which means you can roll
by date and maintain the specified number of backup files.

Usage:

<target type="File" fileName="file.txt"
archiveFileName="archive/file.{#####}.txt" maxArchiveFiles="4"
archiveEvery="Day" archiveNumbering="Sequence|Rolling" />

This creates a new archive file in the archive directory every day.
Depending on archiveNumbering you get a different archive scheme.
Present-day log file is always "file.txt"

a) Sequence

Assuming you've just rolled onto the 101st file, you have:

archive/file.00096.txt
archive/file.00097.txt
archive/file.00098..txt
archive/file.00099.txt
archive/file.00100.txt

(00095 has just been deleted)

b) Rolling

The files are numbered in the opposite direction. The most recent is always
0

archive/file.00000.txt
archive/file.00001.txt
archive/file.00002.txt
archive/file.00003.txt
archive/file.00004.txt

Each time you pass the midnight, files are rolled:

day 0004 is deleted, 3 is moved to 4, 2 to 3, 1 to 2, 0 to 1 and file.txt is
moved to file.00000.txt

The good news it works with any fancy filenaming scheme you may invent. You
may use layout renderers in both file and archive names. The only
requirement is that archives for a single filename are kept in a single
directory.

<target type="File" fileName="file.${logger}.${level}.txt"
archiveName="archives/${level}/${logger}.logfile.{###}" />

Where you actually split one target into many files simultaneously being
written to.
The bad news is that archiving may slow file logging by as much as 30%, but
there are some tricks to overcome it.

2) There's no compression support at the moment, but it should not be hard
too to add this feature (basically just replace File.Delete() with something
different). The trick is to do this in a manageable way, so that ordinary
mortals can use it without too much help. I this we should use .NET 2.0
specific compression classes, ICSharpCode.SharpZipLib or external gzip.exe
or bzip2.exe

In general I think we should have the following options for handling files
that are to be removed from the archive:

Delete - delete the file
Compress - compress using ZIP or GZ
Email - email the file
Exec - spawn an external process passing the name of the file as a parameter

I think this will go to NLog 1.1, as I don't want to add any new features
before 1.0.

The auto-archiving code needs some serious testing, as this is relatively
new feature. I'd like to hear your impressions from testing it. Any feedback
is welcome.

Jarek

----- Original Message -----
From: "Michael Schall" <[hidden email]>
To: <[hidden email]>
Sent: Tuesday, May 30, 2006 6:05 PM
Subject: [Nlog-list] Compress & Delete old logs


> Is there a way with the file target to have it compress itself as it
> is rolling to the next file?  I would name my logs by date, and would
> like the old files to compress (zip, cab, some format readable without
> extra installs) as a new file is being created.
>
> Also, is it possible to have a max number of files saved?  We would
> like to only keep the last x days of logs.
>
> So day to day the files would be compressed and after x days, the
> compressed files would be removed.
>
> Is this possible currently, or planned in the future?
>
> Mike
>
>
> -------------------------------------------------------
> All the advantages of Linux Managed Hosting--Without the Cost and Risk!
> Fully trained technicians. The highest number of Red Hat certifications in
> the hosting industry. Fanatical Support. Click to learn more
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
> _______________________________________________
> Nlog-list mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/nlog-list
>



-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
Nlog-list mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/nlog-list
NLog Blog
Reply | Threaded
Open this post in threaded view
|

Re: Compress & Delete old logs

Jarek Kowalski
Administrator
NLog 1.0 is to be released very soon. I'm just adding some missing
documentation.
I think I'll create a new branch for 1.1 as I've recently seen an explosion
of new ideas...

Would anyone, who's English native speaker, care to take a look at
http://www.nlog-project.org/snapshots/20060528/web/tutorial.html

and report any spelling/grammar issues here?

Jarek
----- Original Message -----
From: "Michael Schall" <[hidden email]>
To: "Jaroslaw Kowalski" <[hidden email]>
Sent: Wednesday, May 31, 2006 2:03 AM
Subject: Re: [Nlog-list] Compress & Delete old logs


> That sounds good... I would like to be able to have at least two steps
> in the handling.  For example I would like to be able to zip anything
> older 2 days old and delete zips older than 14 days.
>
> Is there any timeframe for 1.0 and 1.1?
>
> On 5/30/06, Jaroslaw Kowalski <[hidden email]> wrote:
>> Well, the answer is "kind of",
>>
>> 1) Latest snapshots support auto-archiving feature, which means you can
>> roll
>> by date and maintain the specified number of backup files.
>>
>> Usage:
>>
>> <target type="File" fileName="file.txt"
>> archiveFileName="archive/file.{#####}.txt" maxArchiveFiles="4"
>> archiveEvery="Day" archiveNumbering="Sequence|Rolling" />
>>
>> This creates a new archive file in the archive directory every day.
>> Depending on archiveNumbering you get a different archive scheme.
>> Present-day log file is always "file.txt"
>>
>> a) Sequence
>>
>> Assuming you've just rolled onto the 101st file, you have:
>>
>> archive/file.00096.txt
>> archive/file.00097.txt
>> archive/file.00098..txt
>> archive/file.00099.txt
>> archive/file.00100.txt
>>
>> (00095 has just been deleted)
>>
>> b) Rolling
>>
>> The files are numbered in the opposite direction. The most recent is
>> always
>> 0
>>
>> archive/file.00000.txt
>> archive/file.00001.txt
>> archive/file.00002.txt
>> archive/file.00003.txt
>> archive/file.00004.txt
>>
>> Each time you pass the midnight, files are rolled:
>>
>> day 0004 is deleted, 3 is moved to 4, 2 to 3, 1 to 2, 0 to 1 and file.txt
>> is
>> moved to file.00000.txt
>>
>> The good news it works with any fancy filenaming scheme you may invent.
>> You
>> may use layout renderers in both file and archive names. The only
>> requirement is that archives for a single filename are kept in a single
>> directory.
>>
>> <target type="File" fileName="file.${logger}.${level}.txt"
>> archiveName="archives/${level}/${logger}.logfile.{###}" />
>>
>> Where you actually split one target into many files simultaneously being
>> written to.
>> The bad news is that archiving may slow file logging by as much as 30%,
>> but
>> there are some tricks to overcome it.
>>
>> 2) There's no compression support at the moment, but it should not be
>> hard
>> too to add this feature (basically just replace File.Delete() with
>> something
>> different). The trick is to do this in a manageable way, so that ordinary
>> mortals can use it without too much help. I this we should use .NET 2.0
>> specific compression classes, ICSharpCode.SharpZipLib or external
>> gzip.exe
>> or bzip2.exe
>>
>> In general I think we should have the following options for handling
>> files
>> that are to be removed from the archive:
>>
>> Delete - delete the file
>> Compress - compress using ZIP or GZ
>> Email - email the file
>> Exec - spawn an external process passing the name of the file as a
>> parameter
>>
>> I think this will go to NLog 1.1, as I don't want to add any new features
>> before 1.0.
>>
>> The auto-archiving code needs some serious testing, as this is relatively
>> new feature. I'd like to hear your impressions from testing it. Any
>> feedback
>> is welcome.
>>
>> Jarek
>>
>> ----- Original Message -----
>> From: "Michael Schall" <[hidden email]>
>> To: <[hidden email]>
>> Sent: Tuesday, May 30, 2006 6:05 PM
>> Subject: [Nlog-list] Compress & Delete old logs
>>
>>
>> > Is there a way with the file target to have it compress itself as it
>> > is rolling to the next file?  I would name my logs by date, and would
>> > like the old files to compress (zip, cab, some format readable without
>> > extra installs) as a new file is being created.
>> >
>> > Also, is it possible to have a max number of files saved?  We would
>> > like to only keep the last x days of logs.
>> >
>> > So day to day the files would be compressed and after x days, the
>> > compressed files would be removed.
>> >
>> > Is this possible currently, or planned in the future?
>> >
>> > Mike
>> >
>> >
>> > -------------------------------------------------------
>> > All the advantages of Linux Managed Hosting--Without the Cost and Risk!
>> > Fully trained technicians. The highest number of Red Hat certifications
>> > in
>> > the hosting industry. Fanatical Support. Click to learn more
>> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
>> > _______________________________________________
>> > Nlog-list mailing list
>> > [hidden email]
>> > https://lists.sourceforge.net/lists/listinfo/nlog-list
>> >
>>
>>
>



-------------------------------------------------------
All the advantages of Linux Managed Hosting--Without the Cost and Risk!
Fully trained technicians. The highest number of Red Hat certifications in
the hosting industry. Fanatical Support. Click to learn more
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=107521&bid=248729&dat=121642
_______________________________________________
Nlog-list mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/nlog-list
NLog Blog