Very long tests with huge JTL log file

I know none.

But what the problem with huge files? Why you don't like them?

-----
--
Andrey Pohilko
***@GC Maintainer
--
View this message in context: http://jmeter.512774.n5.nabble.com/Very-long-tests-with-huge-JTL-log-file-tp4783083p4783110.html
Sent from the JMeter - User mailing list archive at Nabble.com.

Nermin CALUK

2011-09-08 16:37:24 UTC

First ideas that come to my mind:
- split your test into, say, 5 consecutive threads, each one doing 20% of your test and writing into a separate file OR experiment with a nested loop to achieve this (you can use timestamp variable as a part of the name of your file, or you can use counter if you use loops)
- select less options to log in Configuration
- use CSV instead of XML logging, if possible
- write ERRORS in one file, SUCCESSES in another file

From my experience, huge files won't be a problem if you run JMeter from command line (and if you have sufficient disk space of course :)

Cheers,
Nermin

----- Original Message -----

Hello,
I am going to execute a tests for 3 days, it will made me to have a very
long JTL file. DO you know anyway to rotate the log JTL file ?
Toni.

Oliver Lloyd

2011-09-08 17:32:18 UTC

I've run tests like this resulting in jtl files greater than 25GB, I used the
method mentioned by Nermin, running over 20 servers each running a test that
was configured to execute only 5% of my target load. Works absolutely fine.
(Note. Amazon is your friend for such shenanigans.)

Obviously: use CSV mode, don't log responses, run on the command line.

If you end up with data this big then you really need to be aggregating the
files in situe before trying to download them and I didn't bother trying to
view them locally in the JM GUI either. Instead I loaded the data into mySQL
and played with it from there. There's some pretty cool open source ETL
tools to help with this.

Also, we've recently started playing with mongo for super big datasets -
man, it is fast.

-----
http://www.http503.com/
--
View this message in context: http://jmeter.512774.n5.nabble.com/Very-long-tests-with-huge-JTL-log-file-tp4783083p4783329.html
Sent from the JMeter - User mailing list archive at Nabble.com.

Deepak Shetty

2011-09-08 17:35:49 UTC

Post by Toni Menendez Lopez
DO you know anyway to rotate the log JTL file ?

ditto to what everyone else is saying - you dont want to do this.
just load it up into your favorite rdbms or OLAP tool and analyse from
there.

Post by Toni Menendez Lopez
Hello,
I am going to execute a tests for 3 days, it will made me to have a very
long JTL file. DO you know anyway to rotate the log JTL file ?
Toni.

sebb

2011-09-08 17:46:22 UTC

If you want to get a rough idea of the test performance, you can add a
Summariser which will log a summary line (to jmeter.log) every so
often - 3 mins by default.

This works also in non-GUI mode.

Post by Toni Menendez Lopez
DO you know anyway to rotate the log JTL file ?

ditto to what everyone else is saying - you dont want to do this.
just load it up into your favorite rdbms or OLAP tool and analyse from
there.

Post by Toni Menendez Lopez
Hello,
I am going to execute a tests for 3 days, it will made me to have a very
long JTL file. DO you know anyway to rotate the log JTL file ?
Toni.

Shay Ginsbourg

2011-09-08 17:49:50 UTC

Adding a summary line (to jmeter.log) is an interesting option.
How is the "Summariser" specified in a script?

*Shay Ginsbourg*
Regulatory & Testing Affairs Consultant

Formerly QA Manager of LoadRunner at Mercury Interactive

M.Sc. cum laude in Bio-Medical Engineering
M.Sc. in Mechanical Engineering

*Work:* 035185873
*Mobile:* 0546690915

*Email:* ***@gmail.com

*http://il.linkedin.com/in/shayginsbourg*

*GINSBOURG.COM* <http://www.ginsbourg.com/>
------------------------------

*P** **Please consider your environmental responsibility before printing
this e-mail*.

Post by sebb
If you want to get a rough idea of the test performance, you can add a
Summariser which will log a summary line (to jmeter.log) every so
often - 3 mins by default.
This works also in non-GUI mode.

Post by Toni Menendez Lopez
DO you know anyway to rotate the log JTL file ?

ditto to what everyone else is saying - you dont want to do this.
just load it up into your favorite rdbms or OLAP tool and analyse from
there.

Post by Toni Menendez Lopez
Hello,
I am going to execute a tests for 3 days, it will made me to have a very
long JTL file. DO you know anyway to rotate the log JTL file ?
Toni.

---------------------------------------------------------------------

sebb

2011-09-08 17:55:32 UTC

Post by Shay Ginsbourg
Adding a summary line (to jmeter.log) is an interesting option.
How is the "Summariser" specified in a script?

http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Generate_Summary_Results

Post by Shay Ginsbourg
*Shay Ginsbourg*
Regulatory & Testing Affairs Consultant
Formerly QA Manager of LoadRunner at Mercury Interactive
M.Sc. cum laude in Bio-Medical Engineering
M.Sc. in Mechanical Engineering
*Work:* 035185873
*Mobile:* 0546690915
*http://il.linkedin.com/in/shayginsbourg*
*GINSBOURG.COM* <http://www.ginsbourg.com/>
------------------------------
*P** **Please consider your environmental responsibility before printing
this e-mail*.

Post by Toni Menendez Lopez
DO you know anyway to rotate the log JTL file ?

ditto to what everyone else is saying - you dont want to do this.
just load it up into your favorite rdbms or OLAP tool and analyse from
there.

Post by Toni Menendez Lopez
Hello,
I am going to execute a tests for 3 days, it will made me to have a very
long JTL file. DO you know anyway to rotate the log JTL file ?
Toni.

---------------------------------------------------------------------

Shay Ginsbourg

2011-09-08 18:21:24 UTC

Fine.
Thanks.

*Shay Ginsbourg*
Regulatory & Testing Affairs Consultant

Formerly QA Manager of LoadRunner at Mercury Interactive

M.Sc. cum laude in Bio-Medical Engineering
M.Sc. in Mechanical Engineering

*Work:* 035185873
*Mobile:* 0546690915

*Email:* ***@gmail.com

*http://il.linkedin.com/in/shayginsbourg*

*GINSBOURG.COM* <http://www.ginsbourg.com/>
------------------------------

*P** **Please consider your environmental responsibility before printing
this e-mail*.

Post by Shay Ginsbourg
Adding a summary line (to jmeter.log) is an interesting option.
How is the "Summariser" specified in a script?

http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Generate_Summary_Results

Post by Toni Menendez Lopez
DO you know anyway to rotate the log JTL file ?

ditto to what everyone else is saying - you dont want to do this.
just load it up into your favorite rdbms or OLAP tool and analyse from
there.
On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez <

Post by Toni Menendez Lopez
Hello,
I am going to execute a tests for 3 days, it will made me to have a

very

Post by Toni Menendez Lopez
long JTL file. DO you know anyway to rotate the log JTL file ?
Toni.

---------------------------------------------------------------------

Tonimenen

2011-09-08 20:43:46 UTC

The idea was doing some kind of rotatelog in JTL, and analyze old files generated with other jmeter in gui mode, and after analyzing droping old JTL files.

Enviado desde mi iPad

Post by Shay Ginsbourg
Fine.
Thanks.
*Shay Ginsbourg*
Regulatory & Testing Affairs Consultant
Formerly QA Manager of LoadRunner at Mercury Interactive
M.Sc. cum laude in Bio-Medical Engineering
M.Sc. in Mechanical Engineering
*Work:* 035185873
*Mobile:* 0546690915
*http://il.linkedin.com/in/shayginsbourg*
*GINSBOURG.COM* <http://www.ginsbourg.com/>
------------------------------
*P** **Please consider your environmental responsibility before printing
this e-mail*.

Post by Shay Ginsbourg
Adding a summary line (to jmeter.log) is an interesting option.
How is the "Summariser" specified in a script?

http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Generate_Summary_Results

Post by Toni Menendez Lopez
DO you know anyway to rotate the log JTL file ?

ditto to what everyone else is saying - you dont want to do this.
just load it up into your favorite rdbms or OLAP tool and analyse from
there.
On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez <

Post by Toni Menendez Lopez
Hello,
I am going to execute a tests for 3 days, it will made me to have a

very

Post by Toni Menendez Lopez
long JTL file. DO you know anyway to rotate the log JTL file ?
Toni.

---------------------------------------------------------------------

Adrian Speteanu

2011-10-18 18:07:26 UTC

This is what I always wanted JMeter to do out of the box, if it supports
log4j or slf4j, so that you don't always have long setup times before you
start a test (there are tons of other things you have to consider, except
how often do you need the files to be splitted).

For that matter, does anyone know of a Linux / Unix utility that can help
with this? (might also be useful to learn to have available for other apps
as well you know).

Post by Tonimenen
The idea was doing some kind of rotatelog in JTL, and analyze old files
generated with other jmeter in gui mode, and after analyzing droping old JTL
files.
Enviado desde mi iPad

Post by Shay Ginsbourg
Adding a summary line (to jmeter.log) is an interesting option.
How is the "Summariser" specified in a script?

http://jakarta.apache.org/jmeter/usermanual/component_reference.html#Generate_Summary_Results

printing

Post by Shay Ginsbourg
this e-mail*.

Post by Toni Menendez Lopez
DO you know anyway to rotate the log JTL file ?

ditto to what everyone else is saying - you dont want to do this.
just load it up into your favorite rdbms or OLAP tool and analyse

from

Post by Deepak Shetty
there.
On Thu, Sep 8, 2011 at 9:26 AM, Toni Menendez Lopez <

Post by Toni Menendez Lopez
Hello,
I am going to execute a tests for 3 days, it will made me to have a

very