Discussion:
Mismatch between numbers in Aggregate Report and InfluxDB via backend listener
Paul Power
2017-05-05 13:16:53 UTC
Permalink
Hi:

In JMeter 3.2 I am running a simple HTTP POST json test on my own PC
against processes also running on my PC. In total 150 requests are sent.

I am still developing the test so I have both an Aggregate Report and
Backend listener.

My problem is that the numbers do not match e.g.

Average is 1566ms in Aggregate Report and 1649 in InfluxDB.
The max is 1850ms in both, but the min is 147 in Aggregate Report and 1122
in InfluxDB.

Is this a problem or do the number have different meanings?

UIltimately I want to disable/remove the Aggregate Report.

​All help gratefully received
,
Paul
--
*The information in this e-mail and any attachments is confidential. It is
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and any
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views expressed in
this message are those of the sender unless expressly stated to be those of
OpenJaw. Virus checking of emails and attachments is the responsibility of
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge House, 1
Ormond Quay Upper, Dublin 7.*
Maxime Chassagneux
2017-05-05 15:26:35 UTC
Permalink
Hi,

Try to change the window mode of the backend metric to time boxed ( i.e in
jmeter.properties and see "BackendListener - configuration" section ).
So each 5 seconds, all metrics are send to influxdb and reset. ( no
aggregation in a slidding window array )

--
Maxime
Post by Paul Power
In JMeter 3.2 I am running a simple HTTP POST json test on my own PC
against processes also running on my PC. In total 150 requests are sent.
I am still developing the test so I have both an Aggregate Report and
Backend listener.
My problem is that the numbers do not match e.g.
Average is 1566ms in Aggregate Report and 1649 in InfluxDB.
The max is 1850ms in both, but the min is 147 in Aggregate Report and 1122
in InfluxDB.
Is this a problem or do the number have different meanings?
UIltimately I want to disable/remove the Aggregate Report.
​All help gratefully received
,
Paul
--
*The information in this e-mail and any attachments is confidential. It is
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and any
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views expressed in
this message are those of the sender unless expressly stated to be those of
OpenJaw. Virus checking of emails and attachments is the responsibility of
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge House, 1
Ormond Quay Upper, Dublin 7.*
Paul Power
2017-05-08 07:03:07 UTC
Permalink
Many thanks Maxime:

I made these changes:

backend_metrics_window_mode=timed
backend_metrics_large_window=50000

The average, max and min are now correct. The 90%, 95% and 99% values are
still wrong.

Regards,
Paul
Post by Maxime Chassagneux
Hi,
Try to change the window mode of the backend metric to time boxed ( i.e in
jmeter.properties and see "BackendListener - configuration" section ).
So each 5 seconds, all metrics are send to influxdb and reset. ( no
aggregation in a slidding window array )
--
Maxime
Post by Paul Power
In JMeter 3.2 I am running a simple HTTP POST json test on my own PC
against processes also running on my PC. In total 150 requests are sent.
I am still developing the test so I have both an Aggregate Report and
Backend listener.
My problem is that the numbers do not match e.g.
Average is 1566ms in Aggregate Report and 1649 in InfluxDB.
The max is 1850ms in both, but the min is 147 in Aggregate Report and
1122
Post by Paul Power
in InfluxDB.
Is this a problem or do the number have different meanings?
UIltimately I want to disable/remove the Aggregate Report.
​All help gratefully received
,
Paul
--
*The information in this e-mail and any attachments is confidential. It
is
Post by Paul Power
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and
any
Post by Paul Power
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views expressed
in
Post by Paul Power
this message are those of the sender unless expressly stated to be those
of
Post by Paul Power
OpenJaw. Virus checking of emails and attachments is the responsibility
of
Post by Paul Power
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge
House, 1
Post by Paul Power
Ormond Quay Upper, Dublin 7.*
--
*The information in this e-mail and any attachments is confidential. It is
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and any
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views expressed in
this message are those of the sender unless expressly stated to be those of
OpenJaw. Virus checking of emails and attachments is the responsibility of
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge House, 1
Ormond Quay Upper, Dublin 7.*
Maxime Chassagneux
2017-05-08 09:47:59 UTC
Permalink
Hi,

Perfect for mean, max and min metrics ! Now for percentile, timeboxed don't
makes sense, so we use a large sliding window ( 50 000 as you set ).
I have to check in report what method is use to percentile.

The difference is important in your result ?
Post by Paul Power
backend_metrics_window_mode=timed
backend_metrics_large_window=50000
The average, max and min are now correct. The 90%, 95% and 99% values are
still wrong.
Regards,
Paul
Post by Maxime Chassagneux
Hi,
Try to change the window mode of the backend metric to time boxed ( i.e
in
Post by Maxime Chassagneux
jmeter.properties and see "BackendListener - configuration" section ).
So each 5 seconds, all metrics are send to influxdb and reset. ( no
aggregation in a slidding window array )
--
Maxime
Post by Paul Power
In JMeter 3.2 I am running a simple HTTP POST json test on my own PC
against processes also running on my PC. In total 150 requests are
sent.
Post by Maxime Chassagneux
Post by Paul Power
I am still developing the test so I have both an Aggregate Report and
Backend listener.
My problem is that the numbers do not match e.g.
Average is 1566ms in Aggregate Report and 1649 in InfluxDB.
The max is 1850ms in both, but the min is 147 in Aggregate Report and
1122
Post by Paul Power
in InfluxDB.
Is this a problem or do the number have different meanings?
UIltimately I want to disable/remove the Aggregate Report.
​All help gratefully received
,
Paul
--
*The information in this e-mail and any attachments is confidential. It
is
Post by Paul Power
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and
any
Post by Paul Power
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views
expressed
Post by Maxime Chassagneux
in
Post by Paul Power
this message are those of the sender unless expressly stated to be
those
Post by Maxime Chassagneux
of
Post by Paul Power
OpenJaw. Virus checking of emails and attachments is the responsibility
of
Post by Paul Power
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge
House, 1
Post by Paul Power
Ormond Quay Upper, Dublin 7.*
--
*The information in this e-mail and any attachments is confidential. It is
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and any
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views expressed in
this message are those of the sender unless expressly stated to be those of
OpenJaw. Virus checking of emails and attachments is the responsibility of
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge House, 1
Ormond Quay Upper, Dublin 7.*
Paul Power
2017-05-08 11:09:55 UTC
Permalink
90%: 1593.9 in InfluxDb, 1735 in Aggregate Report
95%: 1631.9 in InfluxDb, 2542 in Aggregate Report
99%: 1646.93 in InfluxDB, 3256 in Aggregate Report

Max is 3622 for this run

Regards,
Paul
Post by Maxime Chassagneux
Hi,
Perfect for mean, max and min metrics ! Now for percentile, timeboxed don't
makes sense, so we use a large sliding window ( 50 000 as you set ).
I have to check in report what method is use to percentile.
The difference is important in your result ?
Post by Paul Power
backend_metrics_window_mode=timed
backend_metrics_large_window=50000
The average, max and min are now correct. The 90%, 95% and 99% values
are
Post by Paul Power
still wrong.
Regards,
Paul
Post by Maxime Chassagneux
Hi,
Try to change the window mode of the backend metric to time boxed (
i.e
Post by Paul Power
in
Post by Maxime Chassagneux
jmeter.properties and see "BackendListener - configuration" section ).
So each 5 seconds, all metrics are send to influxdb and reset. ( no
aggregation in a slidding window array )
--
Maxime
Post by Paul Power
In JMeter 3.2 I am running a simple HTTP POST json test on my own PC
against processes also running on my PC. In total 150 requests are
sent.
Post by Maxime Chassagneux
Post by Paul Power
I am still developing the test so I have both an Aggregate Report and
Backend listener.
My problem is that the numbers do not match e.g.
Average is 1566ms in Aggregate Report and 1649 in InfluxDB.
The max is 1850ms in both, but the min is 147 in Aggregate Report and
1122
Post by Paul Power
in InfluxDB.
Is this a problem or do the number have different meanings?
UIltimately I want to disable/remove the Aggregate Report.
​All help gratefully received
,
Paul
--
*The information in this e-mail and any attachments is confidential.
It
Post by Paul Power
Post by Maxime Chassagneux
is
Post by Paul Power
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message
and
Post by Paul Power
Post by Maxime Chassagneux
any
Post by Paul Power
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views
expressed
Post by Maxime Chassagneux
in
Post by Paul Power
this message are those of the sender unless expressly stated to be
those
Post by Maxime Chassagneux
of
Post by Paul Power
OpenJaw. Virus checking of emails and attachments is the
responsibility
Post by Paul Power
Post by Maxime Chassagneux
of
Post by Paul Power
the recipient and OpenJaw cannot accept responsibility for any loss
or
Post by Paul Power
Post by Maxime Chassagneux
Post by Paul Power
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland
having
Post by Paul Power
Post by Maxime Chassagneux
Post by Paul Power
registration number 353613 and registered office at Grattan Bridge
House, 1
Post by Paul Power
Ormond Quay Upper, Dublin 7.*
--
*The information in this e-mail and any attachments is confidential. It
is
Post by Paul Power
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and
any
Post by Paul Power
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views expressed
in
Post by Paul Power
this message are those of the sender unless expressly stated to be those
of
Post by Paul Power
OpenJaw. Virus checking of emails and attachments is the responsibility
of
Post by Paul Power
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge
House, 1
Post by Paul Power
Ormond Quay Upper, Dublin 7.*
--
*The information in this e-mail and any attachments is confidential. It is
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and any
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views expressed in
this message are those of the sender unless expressly stated to be those of
OpenJaw. Virus checking of emails and attachments is the responsibility of
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge House, 1
Ormond Quay Upper, Dublin 7.*
Philippe Mouawad
2017-05-08 11:38:29 UTC
Permalink
Hello,
Can you compare with Report Dashboard summary results:

- http://jmeter.apache.org/usermanual/generating-dashboard.html

Regarding values in InfluxDB, how are you computing them ?

Thanks
Post by Paul Power
90%: 1593.9 in InfluxDb, 1735 in Aggregate Report
95%: 1631.9 in InfluxDb, 2542 in Aggregate Report
99%: 1646.93 in InfluxDB, 3256 in Aggregate Report
Max is 3622 for this run
Regards,
Paul
Post by Maxime Chassagneux
Hi,
Perfect for mean, max and min metrics ! Now for percentile, timeboxed
don't
Post by Maxime Chassagneux
makes sense, so we use a large sliding window ( 50 000 as you set ).
I have to check in report what method is use to percentile.
The difference is important in your result ?
Post by Paul Power
backend_metrics_window_mode=timed
backend_metrics_large_window=50000
The average, max and min are now correct. The 90%, 95% and 99% values
are
Post by Paul Power
still wrong.
Regards,
Paul
On 5 May 2017 at 16:26, Maxime Chassagneux <
Post by Maxime Chassagneux
Hi,
Try to change the window mode of the backend metric to time boxed (
i.e
Post by Paul Power
in
Post by Maxime Chassagneux
jmeter.properties and see "BackendListener - configuration" section
).
Post by Maxime Chassagneux
Post by Paul Power
Post by Maxime Chassagneux
So each 5 seconds, all metrics are send to influxdb and reset. ( no
aggregation in a slidding window array )
--
Maxime
Post by Paul Power
In JMeter 3.2 I am running a simple HTTP POST json test on my own
PC
Post by Maxime Chassagneux
Post by Paul Power
Post by Maxime Chassagneux
Post by Paul Power
against processes also running on my PC. In total 150 requests are
sent.
Post by Maxime Chassagneux
Post by Paul Power
I am still developing the test so I have both an Aggregate Report
and
Post by Maxime Chassagneux
Post by Paul Power
Post by Maxime Chassagneux
Post by Paul Power
Backend listener.
My problem is that the numbers do not match e.g.
Average is 1566ms in Aggregate Report and 1649 in InfluxDB.
The max is 1850ms in both, but the min is 147 in Aggregate Report
and
Post by Maxime Chassagneux
Post by Paul Power
Post by Maxime Chassagneux
1122
Post by Paul Power
in InfluxDB.
Is this a problem or do the number have different meanings?
UIltimately I want to disable/remove the Aggregate Report.
​All help gratefully received
,
Paul
--
*The information in this e-mail and any attachments is
confidential.
Post by Maxime Chassagneux
It
Post by Paul Power
Post by Maxime Chassagneux
is
Post by Paul Power
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message
and
Post by Paul Power
Post by Maxime Chassagneux
any
Post by Paul Power
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views
expressed
Post by Maxime Chassagneux
in
Post by Paul Power
this message are those of the sender unless expressly stated to be
those
Post by Maxime Chassagneux
of
Post by Paul Power
OpenJaw. Virus checking of emails and attachments is the
responsibility
Post by Paul Power
Post by Maxime Chassagneux
of
Post by Paul Power
the recipient and OpenJaw cannot accept responsibility for any loss
or
Post by Paul Power
Post by Maxime Chassagneux
Post by Paul Power
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland
having
Post by Paul Power
Post by Maxime Chassagneux
Post by Paul Power
registration number 353613 and registered office at Grattan Bridge
House, 1
Post by Paul Power
Ormond Quay Upper, Dublin 7.*
--
*The information in this e-mail and any attachments is confidential. It
is
Post by Paul Power
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and
any
Post by Paul Power
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views
expressed
Post by Maxime Chassagneux
in
Post by Paul Power
this message are those of the sender unless expressly stated to be
those
Post by Maxime Chassagneux
of
Post by Paul Power
OpenJaw. Virus checking of emails and attachments is the responsibility
of
Post by Paul Power
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge
House, 1
Post by Paul Power
Ormond Quay Upper, Dublin 7.*
--
*The information in this e-mail and any attachments is confidential. It is
intended solely for the addressee(s) named above. If you are not an
intended recipient, please notify the sender and delete the message and any
attachments from your system. Any use, copying or disclosure of the
contents is unauthorised unless expressly permitted. Any views expressed in
this message are those of the sender unless expressly stated to be those of
OpenJaw. Virus checking of emails and attachments is the responsibility of
the recipient and OpenJaw cannot accept responsibility for any loss or
damage arising from the use of this email or attachments. OpenJaw
Technologies Limited is a limited company registered in Ireland having
registration number 353613 and registered office at Grattan Bridge House, 1
Ormond Quay Upper, Dublin 7.*
--
Cordialement.
Philippe Mouawad.
Loading...