tag:blogger.com,1999:blog-11796992290819743692024-03-08T11:14:17.045-08:00KeYeR's Code KitchenKeYeRhttp://www.blogger.com/profile/04045888338509932342noreply@blogger.comBlogger7125tag:blogger.com,1999:blog-1179699229081974369.post-19433386265307523402016-01-30T11:22:00.002-08:002016-01-30T11:24:02.320-08:00How to read performance tests report and use it to create reliable SLA (by example)<div style="text-align: justify;">
<div>
<i>Analysis presented in this post are mostly suitable to analyse performance of web pages and services (especially web services) but also can give a hint how to approach performance report analysis of different application types.</i><br />
<br />
If you have configured your performance tests well you will finish with rich report having many cool graphs and numerical data. This post will explain how to work with such a report well. It will also show what data should such a report contain for good application analysis. You can use it to know what extra widgets should be added to your report.<br />
<br />
<a href="http://petermodzelewski.blogspot.com/2014/02/performance-tests-made-easy.html">In the last post</a> I've describe how to create decent performance tests resulting with pretty report. Let's use this <a href="http://loadosophia.org/" target="_blank">loadosophias</a> report as an example of a good performance tests report and describe the parts I find the most useful in everyday work.<br />
<br />
(<i><span style="font-size: x-small;">In screens below I am using old loadsophias report layout as it has all of the graphs on one tab which makes legend below easier. All graphs described below are present on the new layout and can be found easily.</span></i>)</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixeD7RjzIt6Q_g278UInuEQrEVDcEPEm1_bCAXbAK3wCSK2COaXzPlQDtP42PAiMYIDJYdI9XPtJTpt53gJU6U_iR0h8AksqM8OMGaV6uEumdjHvEJKpEn874b1gQPmjMZ6IbB9kUGD28/s1600/525521eaf92ea154d40000d8.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixeD7RjzIt6Q_g278UInuEQrEVDcEPEm1_bCAXbAK3wCSK2COaXzPlQDtP42PAiMYIDJYdI9XPtJTpt53gJU6U_iR0h8AksqM8OMGaV6uEumdjHvEJKpEn874b1gQPmjMZ6IbB9kUGD28/s400/525521eaf92ea154d40000d8.jpeg" width="376" /></a></div>
<br />
Although the whole report is filled with interesting and important statistical data we should analyse, there are <b>five </b>things that should be analysed first (according to numbers on the picture above).<br />
<br />
<h3>
Summary and why this is not enough for creating a reliable <a href="https://en.wikipedia.org/wiki/Service-level_agreement" target="_blank">SLA</a></h3>
<ol><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFveT9H_d2sJT0eKkPzrOwNXTXzCLsqijMqtVTL13sPqMwfgDQaU8Pvex0B68u71SdubDbWiCE1UgDkZyBHHz7jjW8gxkpL4yBtf-Wzj4VxAnM95oXv7wfR2-UYeKRDWXfQclwX5HS7CI/s1600/1.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="176" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjFveT9H_d2sJT0eKkPzrOwNXTXzCLsqijMqtVTL13sPqMwfgDQaU8Pvex0B68u71SdubDbWiCE1UgDkZyBHHz7jjW8gxkpL4yBtf-Wzj4VxAnM95oXv7wfR2-UYeKRDWXfQclwX5HS7CI/s320/1.png" width="320" /></a></ol>
Test summery information <b><span style="color: red;">(1)</span></b> is the data you will normally end after running tests with typical test tools and should only be a start in well configured report. Those information are only really useful to compare the results to previous runs. It gives some overview of the system, although it doesn't explain why those values are as such - so it is <b>not sufficient </b>to create an <a href="https://en.wikipedia.org/wiki/Service-level_agreement" target="_blank">SLA</a>. Why? Such document should be a contract about what conditions we <b>guarantee</b> to client and with what probability.</div>
<div style="text-align: justify;">
For example SLA should state that we ensure that 99% of requests will end in at most 100 ms. Saying that avg response time will be 100 ms says nothing, because it can mean that 50% will be 190 ms and 50% will be 10 ms. If client would then set timeout to 110 ms he will get problems in 50% of cases - which is in most cases unacceptable and against SLA we've agreed to.</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: justify;">
Response time diagnosis</h3>
<div style="text-align: justify;">
To create a meaningful SLA or to check application performance in case of response time we need to see how those values were behaving through the time of test and what is its distribution.</div>
<div style="text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_MY7VQq9oJDmx10i7Eak2xjHDBofxMcmHrdq9B0s-msNpkqYGt4JdAjTP_vGEOJJb99WLapZC3hdHulw0TeXwi1HhyphenhypheneDDgLUxADqS66nDm9v6BYNyODc79rRK0yabD8rCeAjdxmFZ7oM/s1600/2.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="220" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_MY7VQq9oJDmx10i7Eak2xjHDBofxMcmHrdq9B0s-msNpkqYGt4JdAjTP_vGEOJJb99WLapZC3hdHulw0TeXwi1HhyphenhypheneDDgLUxADqS66nDm9v6BYNyODc79rRK0yabD8rCeAjdxmFZ7oM/s320/2.png" width="320" /></a></div>
<div style="text-align: justify;">
First big input for creating an SLA is <i>Overall Response time distribution & Quantiles</i> section <b><span style="color: red;">(2)</span></b>. This graph can literally fill our "<i>X% will end in Y ms</i>" section of such a document.</div>
<div style="text-align: justify;">
For example basing on our test, Google could say that <i>90% requests will end in 400 ms</i>. Or that <i>87% of requests will end in 300 ms</i>. Or even, <i>92% of requests will end in 500 ms, but 50% will end in less than 250 ms</i>. </div>
<div style="text-align: justify;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYi1xA_KWPquMW4eL6jpGIt3U3ZEj2GQYTdihuvjjQTDR9W_MMS3WKE3vjofJZKSIpo6F4neXn8OI8JKtMAxzm2MleuIwZgL-p_UqRQCoAGNRfYJ0X5rBV-kf3H5zcf1usDnVgX2ERDiw/s1600/3.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="219" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYi1xA_KWPquMW4eL6jpGIt3U3ZEj2GQYTdihuvjjQTDR9W_MMS3WKE3vjofJZKSIpo6F4neXn8OI8JKtMAxzm2MleuIwZgL-p_UqRQCoAGNRfYJ0X5rBV-kf3H5zcf1usDnVgX2ERDiw/s320/3.png" width="320" /></a>The question of how to write it in documents itself depends on the client profile. This data lets you to adjust SLA for the clients needs, but also gives you important information about your application performance.<br />
In this case one could ask why 7.5% of requests are so far from the general distribution. Most requests end in 200 - 400 ms here, but the other group suddenly take 500 - 600 ms. Such a big deviation can mean application, hardware or even net problems. It's not rare that diagnosing such fact can improve general performance, not only performance of 7.5% requests. </div>
<div style="text-align: justify;">
I'm not saying that you must fight about performance of those 7.5% of requests. maybe even 600 ms is absolutely acceptable for your application? However you should always be able to answer <i><b>why</b> </i>those 7.5% have that deviation in response time.</div>
<div style="text-align: justify;">
Additional input in response time analysis brings <i>Response Times Distribution Over Time</i> graph <b><span style="color: red;">(3)</span></b>. It can additionally say when (according to test time) response time deviations happened. In our example we can see that the long response times appeared at the start of the test and than diapered. It can be an important tip, that whatever was the cause of such response times it disappered over time or is only appearing from time to time.</div>
<div style="text-align: justify;">
<br /></div>
<h3 style="text-align: justify;">
Response codes for the rescue</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLz7GaHkuSHBCrcuzLp89iWJw_3oxo99wg25nLWknD2_zZzdBwxXuR2-wLBF4_jhJ5K8GHLYrJfnDE9VmKkAfDxlJ3d-tkHbGv52QxGydPOIEckwa9q5Sn202eX0_BjXOhl5pml78Ybww/s1600/4.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLz7GaHkuSHBCrcuzLp89iWJw_3oxo99wg25nLWknD2_zZzdBwxXuR2-wLBF4_jhJ5K8GHLYrJfnDE9VmKkAfDxlJ3d-tkHbGv52QxGydPOIEckwa9q5Sn202eX0_BjXOhl5pml78Ybww/s320/4.png" width="320" /></a></div>
<div style="text-align: justify;">
<i>Response code over time graph</i><span style="color: red;"> </span><b><span style="color: red;">(4)</span> </b>is a simple, but powerful tool for diagnosing web applications. First it can tell if your application is working correctly (200 response code). If error pages appear, it will tell you when. It also should be compared to transactions per second (TPS), which can be really helpful.</div>
<div style="text-align: justify;">
For example we can imagine that errors (for example code 500) can appear when test exceeds 15 TPS and it overloads our system (for example backed timeouts). </div>
<div style="text-align: justify;">
In other case our application can experience occasional error codes. This can represent temporary backend problems or web instability. This can sometimes be absolutely acceptable, but sometimes can be a prediction of future troubles. Therefore again it is really important to know <i><b>why</b></i> those occasional errors appear to predict if they will grow and at what scale.<br />
If we predict that some % response codes would be errors (for example when we do some cyclic backend operation that will make our services unavailable for couple of hours) we need to write it in our SLA also.<br />
Measuring response codes is extremely important in web applications performance tests and unfortunately often ignored. I've seen more then once a situation when developers measured performance of an application and were extremely happy of an unexpected performance grow. They didn't notice that it was caused by a big percentage of errors caused by backed malfunction. Simple report they were using (avg TPS and response time) had no indication of response code problems.<br />
<br /></div>
<div style="-webkit-text-stroke-width: 0px; font-family: 'Times New Roman'; font-size: medium; font-variant: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: justify; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">
<h3 style="font-style: normal;">
The story of TPS</h3>
<div>
<div style="font-style: normal;">
<i>Transactions per second graph</i> <b style="font-style: normal;"><span style="color: red;">(5)</span></b> is the cherry on top of those graphs. </div>
<div style="font-style: normal;">
<br /></div>
<div class="separator" style="clear: both; font-style: normal; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk0-5bwXiL9icKlBUcE2EVdZ724X7ZowLG1PhFcRS63eX4XJBri5PC3PJII-8HoFoqxXQHV2YroYnuZV0g4GA2N8sUj0ov_cfnbeXhqmHLKwY_MDLe8g8FMPBJFM2LseYLNLGUJgD3uks/s1600/5.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk0-5bwXiL9icKlBUcE2EVdZ724X7ZowLG1PhFcRS63eX4XJBri5PC3PJII-8HoFoqxXQHV2YroYnuZV0g4GA2N8sUj0ov_cfnbeXhqmHLKwY_MDLe8g8FMPBJFM2LseYLNLGUJgD3uks/s1600/5.jpg" width="320" /></a></div>
<div style="font-style: normal;">
In most cases SLA covers not only response times, but also is stating how many concurrent requests is our application able to handle in any given time.<br />
For such SLA basing on this graph we need to pick a safe number. In this example probably 8-10 TPS would be a safe pick (having in mind that its end is just caused by threads finishing up testing).<br />
Remember to <i><b>not over-advertise</b></i> your service. If you guarantee 9 TPS and the customer gets 15 he will be positively surprised and might hold you in high regard for quality of service. On the other hand if you would promise 15 TPS and deliver 9 you would probably deal with lots of bug reports and customer dissatisfaction.<br />
<br />
In our example <i>Response Count By Type </i> <b><span style="color: red;">(5) </span></b>gives you additionally information about Virtual Users number at the given time. You can use that and give a big ramp up time in your test: For example you can set the test to run 64 threads, and give a rump up time of 640 seconds - every 10 seconds new virtual user will be added showing you how next concurrent user affects TPS.<br />
<br />
This graph should also be used to determine TPS stability. If the VU number is stable, is TPS stable as well? Its obvious that TPS will be jumping in some range (like +/- 5%). If we would notice some more drastic jumps like working at pace of 100 TPS but suddenly dropping to 5 TPS we would be sure that we deal with something serious (network issues, software optimization, some process killing backend from time to time).<br />
Of course no matter of how wide the range of TPS is you need to ask yourself yet again a simple question <i><b>why</b></i>.</div>
<br />
<h3>
Real treasury</h3>
</div>
<div>
<div style="-webkit-text-stroke-width: 0px; color: black; font-family: 'times new roman'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: justify; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px;">
<div style="margin: 0px;">
The <b>real fun</b> begins, when you create such reports systematically (for example every week on the same hour). Such test should be also run after every big (or not) software or hardware change. Having such historical data will open you a door to grand new possibilities - to observe how changes in application, infrastructure, information stored in backend and even user load affects your application performance. </div>
<div style="margin: 0px;">
<br /></div>
</div>
The important thing here is never to be afraid to ask questions. There should never be any voodoo in good report interpretation. Every team member must know <b><i>why </i></b>those numbers appear and if there are errors to realize if and when they will become a problem<br />
<br />
This report gives you great tools to detect abnormalities of your application. It also gives you realistic data for creating SLA for your clients. </div>
<div>
<br /></div>
<div>
<i>This short article is only a tip of an iceberg of possibilities that good reports after performance tests brings. I hope it will be a good start for you and an inspiration to dig further. </i></div>
</div>
KeYeRhttp://www.blogger.com/profile/04045888338509932342noreply@blogger.com0tag:blogger.com,1999:blog-1179699229081974369.post-15854915667748509532014-02-12T12:37:00.004-08:002015-07-21T00:37:04.913-07:00Performance Tests made easy<div style="text-align: justify;">
<br />
<h3>
Why performance tests?</h3>
When you are working on a project which has a web interface (like web-service, webpage or hosting server) you finally will come to a point you will need to start doing some performance tests. There are several reasons for such a necessity:</div>
<ul>
<li style="text-align: left;">to know how much traffic you can serve without problems</li>
<li style="text-align: justify;">to create realistic <a href="http://pl.wikipedia.org/wiki/Service_Level_Agreement">Service Level Agreement</a> </li>
<li style="text-align: left;">to check how well you scale while adding new servers</li>
<li style="text-align: left;">to make a baseline for performance improvements</li>
</ul>
<div style="text-align: justify;">
Knowing how much traffic your product can handle is very crucial in making a decision if you need to invest time in new functionality to get more users, or in scalability or performance changes to handle the users you already have.</div>
<div style="text-align: justify;">
It is always important to have performance tests before you will attempt any performance related changes. Without baseline from before the changes you will have no idea if your work changed anything. It is not rare that developers are spending weeks on something that in the end happen to be a micro-optimization (or even the product is running slower), because they didn't took something under consideration. To not fallow that scenario you need to check if your optimization idea works. To do that, you need the baseline and performance tests to run again in order to compare.<br />
On the other hand those tests cannot take forever to create. One needs to find a way to create them quickly, so it will not delay product development. If it would take days, some teams might sacrifice the need of such tests (accepting the risks) and ignore it until too late. On the other hand if it would take up to an hour every "mentally stable" developer team would like to do it an be left on the safe side.<br />
<br />
<h3>
Creating performance tests for web interface.</h3>
<div>
While creating performance tests, you need to remember to assure some crucial properties. They must:</div>
<div>
<ul>
<li><b>be easy to rerun - </b>you should be able to run them one after another with no problem on any time of day and night. This implicates tests automatisation. It is always good to ask yourself if you could add your test script to crone on some server, and only see test results after the test run. </li>
<li><b>be configurable</b> - such things as host address, number of threads, number of retries etc must be read from config, so you can easily change them and reuse tests on different environments.</li>
<li><b>have results easy to analyse </b>- dumping results to file or only the simplest statistics calculated iare rarely enough. You need to model tests output in such a way, that it would explain not only <i>what </i>but also <i>why</i>. For example if request duration mean is around 100 ms, does it mean that all requests take around 90 ms - 110 ms, or are the requests around 20 ms but some of them can take even over 10 seconds and this is misleading the statistic.</li>
</ul>
<div>
This tutorial will show how to quickly create <b>configurable</b>, <b>automated</b> tests and how to <b>visualize </b>their results in such a way they are easy to analyse. It will take less than half an hour!<br />
<br /></div>
<h3>
Creating a test</h3>
<div>
Let's test google! Our test will show us performance of <b><span style="font-family: Courier New, Courier, monospace;">http://www.google.pl/search?q=<span style="color: #38761d;">{query}</span></span></b><span style="color: #38761d;"> </span>web interface As we want our test to be easily configurable, we will look at is as <b><span style="font-family: Courier New, Courier, monospace;">http://<span style="color: red;">{host}</span>/search?q=<span style="color: #38761d;">{query}</span></span></b>.</div>
<div>
<br />
First, we need to get newest <a href="http://jmeter.apache.org/download_jmeter.cgi">JMeter</a>. Then we need to download zip of <a href="http://jmeter-plugins.org/downloads/all/">jmeter-plugins</a> (standard set is OK) and copy contents of it's <span style="font-family: Courier New, Courier, monospace;">lib/ext</span> to <span style="font-family: Courier New, Courier, monospace;">apache-jmeter-X.Y\lib\ext.</span> <a href="http://jmeter-plugins.org/wiki/Start/">Jmeter-plugins</a> is a great set of JMeter extenstions.<br />
<br />
Having our JMeter configured with jmeter-plugins, we run it in graphical (default) mode with a script you can find in <span style="font-family: 'Courier New', Courier, monospace;">apache-jmeter-X.Y\bin </span><span style="font-family: inherit;">folder. First we will create a Thread Pool for our tests:</span><br />
<span style="font-family: inherit;"><br />
</span> <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaeNOazJc30W4puuIYsWWbO2_oaIfKfDByiy44cFTXi_anqS4v3XqUuOvN4dqu6IaX5VPZ42TsX-VtgFO4VpStZpCGOQ6SEQZnzIeknFFtmppp9rguej3bigTkiBCDqSnnyBrwPbHa6VE/s1600/step1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiaeNOazJc30W4puuIYsWWbO2_oaIfKfDByiy44cFTXi_anqS4v3XqUuOvN4dqu6IaX5VPZ42TsX-VtgFO4VpStZpCGOQ6SEQZnzIeknFFtmppp9rguej3bigTkiBCDqSnnyBrwPbHa6VE/s400/step1.png" width="400" /></a></div>
<span style="font-family: inherit;"><br />
</span> <br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
We are using <b><span style="font-family: Courier New, Courier, monospace;"><a href="http://jmeter.apache.org/usermanual/functions.html#__P">${__P(name)}</a></span></b> tool to read values from property file, where:</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ul>
<li><b><span style="font-family: Courier New, Courier, monospace;">${__P(<span style="color: red;">test.thread.max</span>)}</span> </b>- number of threads we want to use</li>
<li><b><span style="font-family: Courier New, Courier, monospace;">${__P(<span style="color: red;">test.thread.rampUp</span>)}</span> </b> - how much time to spawn the threads</li>
<li><b><span style="font-family: Courier New, Courier, monospace;">${__P(<span style="color: red;">test.baseCount</span>)}</span> </b>- number of tests each thread will do</li>
</ul>
<div>
Then we will add a sampler for our threads - <a href="http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request">Http Reqest</a>:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZEWnRJoRIyzuBK8s_w7fgSwkFCHPDDRWtczmiWFi7E6LQZNCB1GAUMO-xwSF-4M7BCz0mBw1lXdWgnOsD3dyDVqMsA-PfKCFKP8SudH-QlO1wa5cUwjzRoMMI3JBRwWwHC7ZuRkOTKgk/s1600/step2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="271" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgZEWnRJoRIyzuBK8s_w7fgSwkFCHPDDRWtczmiWFi7E6LQZNCB1GAUMO-xwSF-4M7BCz0mBw1lXdWgnOsD3dyDVqMsA-PfKCFKP8SudH-QlO1wa5cUwjzRoMMI3JBRwWwHC7ZuRkOTKgk/s400/step2.png" width="400" /></a></div>
<div>
<br /></div>
<span style="font-family: inherit;">where we use two additional configurable properties:</span><br />
<ul>
<li><b>${__P(<span style="color: red;">google.host</span>)} - </b>to specify tested host</li>
<li><b>${__P(<span style="color: red;">google.query</span>)}</b> - to specify test query</li>
</ul>
<div>
Lets save our test plan as <span style="font-family: Courier New, Courier, monospace;">google.jmx</span>. Then lets create <span style="font-family: Courier New, Courier, monospace;">google.propertes</span>:</div>
<div>
<pre class="brush: java">test.thread.max=4
test.thread.rampUp=1
test.baseCount=10
google.host=google.pl
google.query=loadosophia
</pre>
<div>
Now we can run our test! Lets do it from console (after all that would be the way crone would run it each night)</div>
<pre class="brush: java">java -jar ApacheJMeter.jar -n -t /path/to/google.jmx -q /path/to/google.propertes
Created the tree successfully using google.jmx
Starting the test @ Fri Oct 04 11:54:10 CEST 2013 (1380880450276)
Waiting for possible shutdown message on port 4445
Tidying up ... @ Fri Oct 04 11:54:20 CEST 2013 (1380880460120)
... end of run</pre>
Tests are run! Lets see the results! If you have any problems You can consult files in the <a href="https://github.com/petermodzelewski/snippets/tree/master/jmeter-loadsophia">repo</a>.<br />
<br />
<h3>
Analysing the results with loadosophia.org</h3>
</div>
</div>
</div>
<div>
<a href="https://loadosophia.org/">Loadosophia</a> is a great site, where you can upload jMeter results and receive a rich graphical report helping to analyse it. It has the <a href="https://loadosophia.org/wiki/help:pricing_policy">Pay What You Want</a> policy so you may test it and pay if you feel like it. All operations there are under https and signed with Your google account, so we may say it's decently safe to use. </div>
<div>
We will cover Loadsophia in details soon, but for now You only need to get <a href="https://loadosophia.org/gui/upload/token/">Your Upload Token</a> and <a href="https://loadosophia.org/gui/DEFAULT/add/">create new project</a> in your <a href="https://loadosophia.org/gui/">workspace</a> named GoogleQueryTest.</div>
<div>
</div>
<div>
Now let's open <span style="font-family: Courier New, Courier, monospace;">google.jmx</span> in jMeter graphical interface and add<a href="http://jmeter-plugins.org/wiki/LoadosophiaUploader/"> Loadosophia.org Uploader</a>:</div>
<div>
<br /></div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwVmRYPLZp1hTwhIXfAajVjJEzXDpkLoDtzx9ssCYUCWcvEr8haGDA7X0YseslfmhKuevEV4YdZG7wrtq2L3mEUiFrwytw6dtIpLKw6JUHw_cBYoWgRJDO2LY0BkFwLU3nq8gS8fyRGQU/s1600/step3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="201" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwVmRYPLZp1hTwhIXfAajVjJEzXDpkLoDtzx9ssCYUCWcvEr8haGDA7X0YseslfmhKuevEV4YdZG7wrtq2L3mEUiFrwytw6dtIpLKw6JUHw_cBYoWgRJDO2LY0BkFwLU3nq8gS8fyRGQU/s400/step3.png" width="400" /></a></div>
<br />
We are changing the project name to the one we've just created: GoogleQueryTest, and we need to paste in the token from <a href="https://loadosophia.org/gui/upload/token/">Your Upload Token</a>, I also like to specify the folder to save in results (relative to where tests are run from) and to name test after the time it was run with <b><span style="font-family: Courier New, Courier, monospace;">${__time(<span style="color: red;">yyyy-MM-dd-HH:mm:ss</span>)}</span></b>. It helps in compareson of many tests over time.<br />
Lets save the file, and open console. Lets create the results directory and run the tests:<br />
<pre class="brush: java">$ mkdir results
$ java -jar ApacheJMeter.jar -n -t /path/to/google.jmx -q /path/to/google.propertes
Created the tree successfully using google.jmx
Starting the test @ Fri Oct 04 14:25:58 CEST 2013 (1380889558228)
Waiting for possible shutdown message on port 4445
Tidying up ... @ Fri Oct 04 14:26:22 CEST 2013 (1380889582299)
... end of run
</pre>
Now, we can see <a href="https://loadosophia.org/examples/86535/">our results</a> (I've made the results public for You to see. Normally it require to be member of project to see it's results).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgOYS8TI0HZB7xJLiczZA-USicTqnfLalZCCgkpuo9j2Qi9RHLQklA9cjQbaS-i7NFenK4l1j3_z7r4ewYaQrEQx52iWXgASbXYfzuFm-i5ElsQ51BIBMArVB9Lp8Yifmw5z99PAYxkNI/s1600/Zrzut+ekranu+2014-02-12+21.32.20.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="183" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgOYS8TI0HZB7xJLiczZA-USicTqnfLalZCCgkpuo9j2Qi9RHLQklA9cjQbaS-i7NFenK4l1j3_z7r4ewYaQrEQx52iWXgASbXYfzuFm-i5ElsQ51BIBMArVB9Lp8Yifmw5z99PAYxkNI/s1600/Zrzut+ekranu+2014-02-12+21.32.20.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3>
What's next</h3>
<div>
This tutorial gives You the basic knowledge how to create performance tests in the manner of minutes. The next post will cover the tips how to read <a href="https://loadosophia.org/">Loadosophia</a> report and how to use it to make Your application better.</div>
<br />
<br /></div>
</div>
<div>
<i style="background-color: #fefdfa; color: #333333; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18px;"><span style="font-size: xx-small;">Graphic courtesy of <a href="http://lasvegassportsperformance.com/performance/">lasvegassportsperformance.com</a></span></i></div>
KeYeRhttp://www.blogger.com/profile/04045888338509932342noreply@blogger.com7tag:blogger.com,1999:blog-1179699229081974369.post-86492942005904094032013-06-02T02:45:00.000-07:002013-06-03T02:07:16.038-07:00Short jhat tutorial: diagnosing OutOfMemoryError by example<a href="http://petermodzelewski.blogspot.com/2013/05/how-to-diagnose-outofmemoryerror-in.html">Last time</a> we've learned what can be the reason of OutOfMemoryErrors and what tool-chain can we use to diagnose it. Today we will learn by example how to use the most important one: <a href="http://docs.oracle.com/javase/6/docs/technotes/tools/share/jhat.html"><b>jhat</b></a>.<br />
I've prepared a <a href="https://github.com/petermodzelewski/snippets/tree/master/OutOfMemoryErrorGenerator">sample project</a> for this exercise, which can be cloned from <a href="https://github.com/petermodzelewski/snippets/tree/master/OutOfMemoryErrorGenerator">github</a>. The code is really simple and its problem is obvious, but this simplicity will make it easy to get to know jhat.<br />
First, we need to run our program. We will use small heap size, for two reasons:<br />
<ol>
<li>Program will throw the exception faster</li>
<li>jhat will start more quickly, as the heap dump will be smaller</li>
</ol>
<pre class="brush: java">$ git clone https://github.com/petermodzelewski/snippets.git
$ cd snippets/OutOfMemoryErrorGenerator/
$ mvn package
$ java -Xmx128m -Xms128m -jar target/OutOfMemoryErrorGenerator-1.0.jar
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at pl.keyer.oome.generator.Sleepyhead(Sleepyhead.java:6)
at pl.keyer.oome.generator.App.main(App.java:11)
</pre>
<br />
We can notice, that the program is still running. We will need another console to run jhat. We are using the following commands:<br />
<pre class="brush: java">$ jps -l
752 target/OutOfMemoryErrorGenerator-1.0.jar
4480 sun.tools.jps.Jps
$ jmap -dump:file=dump.map 752
$ jhat -port 7401 dump.map
Reading from dump.map...
Dump file created Sat Jun 01 23:25:55 CEST 2013
Snapshot read, resolving...
Resolving 561438 objects...
Chasing references, expect 112 dots................................................................................................................
Eliminating duplicate references................................................................................................................
Snapshot resolved.
Started HTTP server on port 7401
Server is ready.
</pre>
<br />
Important notes about that process:<br />
<ul>
<li>All commands must be executed by the same user: the java process owner</li>
<li>The <i>"expect X dots" </i>message is not a joke. While processing bigger heap dumps one can check the number of dots there in editor to see the progress, as it can take quite a while to process such a file.</li>
<li>When processing bigger dumps one must watch heap size of jhat itself. This depends on the case, but to be safe (provided with enough resources) jhat should have 2-4 times more heap size, than process heap it will diagnose. If memory size for jhat is too small it will just crush after using it and the process will need to be repeated with bigger amount of memory. For example to provide jhat with 4 gigs the command will be:<br />
<pre class="brush: java">$ jhat -port 7401 -J-mx4G dump.map</pre>
</li>
<li>Diagnosed process may be terminated after dumping heap with jmap.
</li>
<li>Obviously jhat can be run on any machine where the dump will be present. On many occasions developers choose to zip the dump and move the debug process to machine more accessible for them and with enough ram. </li>
</ul>
After executing the commands we can visit <a href="http://localhost:7401/" target="_blank">http://localhost:7401/</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3YSEfMmeHWlXwH085c-BEXjbVlQpzWhduExA5XZohnSIhUKk-oEvicAQYMIfBrIQb9LNN5MxeskKgbZjzoTcAMHzd3fHrEAZocle9Nzs1VEJiB2rL93v1ZHatoTR0d57pEclYkp-e8BU/s1600/jhat.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3YSEfMmeHWlXwH085c-BEXjbVlQpzWhduExA5XZohnSIhUKk-oEvicAQYMIfBrIQb9LNN5MxeskKgbZjzoTcAMHzd3fHrEAZocle9Nzs1VEJiB2rL93v1ZHatoTR0d57pEclYkp-e8BU/s400/jhat.png" width="400" /></a></div>
When facing with jhat You will quickly realize that this tool is from times, where such tools were designed without consideration of prettiness or usability. This tutorial will show how to navigate it in most cases - all it's features are cool and necessary, but everyday programmer will use only subset of them to quickly diagnose where the OOME came from.<br />
jhat main page can be divided into sections:<br />
<br />
<ol>
<li>List of all classes in your program (excluding platform - that is, all that is not from standard library). This list is normally really long and in most cases it is not necessary. Normally You will scroll down to "Other Queries" section right away. </li>
<li>More options for listing classes</li>
<li>Bread and butter of memory debugging, we will use them in a moment</li>
<li>More tools for debugging, but not as helpfull as section 3.</li>
<ul>
<li>Heap histogram is sometimes useful to compare quantity vs size of objects</li>
<li>When you become jhat ninja, you sometimes could use <a href="http://localhost:7401/oqlhelp/" target="_blank">OQL </a>to diagnose the application. It is a SQL-like language for searching heap and calculating it's statistics. </li>
</ul>
</ol>
<div>
Firstly, lets see <a href="http://localhost:7401/histo/" target="_blank">heap histogram</a>.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhI6B3Y55fpKo48Fg-rhvMHJKXiW4cJm11hWpaMYawEGLBtjV81oYzB8_QKHIsUHPDqCZHAX274JX4UXFP-DSBxapLeR73TENv8OrIIB9ChBeZzmLL_EZXKayJcWnksTV6TcKAd9ZJfdI8/s1600/jhat1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="323" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhI6B3Y55fpKo48Fg-rhvMHJKXiW4cJm11hWpaMYawEGLBtjV81oYzB8_QKHIsUHPDqCZHAX274JX4UXFP-DSBxapLeR73TENv8OrIIB9ChBeZzmLL_EZXKayJcWnksTV6TcKAd9ZJfdI8/s400/jhat1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
This view illustrates very important fact, that jhat does not compute "deep" object size. That's why on top of our process' memory consumption histogram we see<b> class [B</b> which is really an array of bytes: <b><span style="font-family: Courier New, Courier, monospace;">byte[]</span></b>. Array of bytes is often on top of such histogram and what's more - in most cases it doesn't mean that there is a problem. For example, if your program processes lot's of strings it will naturally have lot of byte arrays, as each string object contains reference to some byte array. Similar problem will manifest when we follow the <a href="http://localhost:7401/showInstanceCounts/includePlatform/" target="_blank">"Show instance counts for all classes (including platform)"</a>.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihVJEB12OznJBcMrAfF1Hiyag812vnKgW_2p7yYFT9bZZ3PDEQRwQ7pbT0EELq3MDJCB9YX4703Jc58zeieWg1s9SIebOBcVJIrZkjlQDJ71FAY0blA7_XxaaT91-Md8nrssz2DhEpx6g/s1600/jhat2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="323" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihVJEB12OznJBcMrAfF1Hiyag812vnKgW_2p7yYFT9bZZ3PDEQRwQ7pbT0EELq3MDJCB9YX4703Jc58zeieWg1s9SIebOBcVJIrZkjlQDJ71FAY0blA7_XxaaT91-Md8nrssz2DhEpx6g/s400/jhat2.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
That view is very similar to histogram sorted by quantity. Normally during OOMR we will look for "our" classes. We will need to exclude platform classes to easily detect abnormalities in our classes quantity (or size). A good start is to follow <a href="http://localhost:7401/showInstanceCounts/" target="_blank">"Show instance counts for all classes (excluding platform)"</a>.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEianFiCVE2Um75L3hwCilia3RD4MZ6bAwpi1j_aZO0l8kCNcb-FS_CJwqeg6VcdJ3fY8TD7PDV62pA1Rc4uArtCd9eR58UfN6pg0DKXafbOzhiwi9WoGRV7j0zvKMZb_8qq6fRW9sqS62I/s1600/jhat3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="175" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEianFiCVE2Um75L3hwCilia3RD4MZ6bAwpi1j_aZO0l8kCNcb-FS_CJwqeg6VcdJ3fY8TD7PDV62pA1Rc4uArtCd9eR58UfN6pg0DKXafbOzhiwi9WoGRV7j0zvKMZb_8qq6fRW9sqS62I/s400/jhat3.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
Things to look: <b>unnaturally big numbers of objects of some class</b>. In our example the number of workers. Our example illustrates the common problem, when producer creates tasks faster, than consumer handles them, and the producer - consumer queue will not block after limit is reached. </div>
<div class="separator" style="clear: both; text-align: left;">
Unfortunately in most cases it is not as easy. Diagnosing that objects of some class are eating too much memory is one thing. Diagnosing why and where are they allocated is another. To do so, we need to track the objects references to something "active", for example thread, controller, etc. When given that "active" object we can than analyze the algorithm and find out why so many objects are created. </div>
<div class="separator" style="clear: both; text-align: left;">
To illustrate such process lets track <b>class [B</b> references.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie7dAgsGOD0Re44LBK_ow6o6SWHtRi64t2GGJO6QzMzXD4THNBHvl25h-lGJSkrHfIQo_gfwjrvfb2q6t1djUNTADCh4rNM7s4tNMVOJld2sBHFLXf8cLNL7ZW94cf1fJvExO1RYOxS2I/s1600/jhat4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEie7dAgsGOD0Re44LBK_ow6o6SWHtRi64t2GGJO6QzMzXD4THNBHvl25h-lGJSkrHfIQo_gfwjrvfb2q6t1djUNTADCh4rNM7s4tNMVOJld2sBHFLXf8cLNL7ZW94cf1fJvExO1RYOxS2I/s400/jhat4.png" width="368" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div>
jhat enables going through references as long as one needs. Unfortunately when you click on class "page" it will display all it's instances. To dodge that time consuming view, you can copy link of the class page (for example from histogram view) and construct reference by Type view link. For example <b>class [B</b> page can have the following url:<br />
<span style="font-family: Courier New, Courier, monospace;">http://localhost:7401/<span style="color: #274e13;">class</span>/<b><span style="color: blue;">0x3881d790</span></b></span><br />
so the reference by type summery will have url as follow:<br />
<span style="font-family: Courier New, Courier, monospace;">http://localhost:7401/<b><span style="color: #274e13;">refsByType</span></b>/<b><span style="color: blue;">0x3881d790</span></b></span></div>
<div>
<br /></div>
<div>
Those methods normally are all you need to detect the memory trouble in the process. Additionally in case you are working with application container (like tomcat) and having problems to see what application (or pool) is leaking the objects, you should diagnose the Classloader section of class page:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSnZYm4-9b7e9lhjgp1kKkXqH370pguDag_XtYMEOZBK9ZyY2NY3Zh_ukguHBnirjJwP71rcjy7XeFnCwmmTaxwT_4VfslGtFOPltWITZSlqyKOS0vjUgaorpn9zVjYGjmbyGyDmjcHf8/s1600/jhat5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="302" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSnZYm4-9b7e9lhjgp1kKkXqH370pguDag_XtYMEOZBK9ZyY2NY3Zh_ukguHBnirjJwP71rcjy7XeFnCwmmTaxwT_4VfslGtFOPltWITZSlqyKOS0vjUgaorpn9zVjYGjmbyGyDmjcHf8/s400/jhat5.png" width="400" /></a></div>
<div>
<br /></div>
<div>
That's it. All you need to know about jhat to start your own memory problem debug. Hope it'll help.</div>
<div>
<b><br /></b></div>
<div>
<b>One last tip</b>: Many developers after solving their memory problem while running new application version, are taking heap dump and running jhat with it - just in case. They are often terrified that despite their efforts object count is still grand but somehow OOME is not appearing. Of course they were so focused on fighting the memory leakage that they've forget how GC works. Remember: <b>Always trigger GC collection (for example with <a href="http://visualvm.java.net/">visual vm</a>) to clean old gen from unnecessary objects before taking the dump and analyzing it with jhat. </b></div>
KeYeRhttp://www.blogger.com/profile/04045888338509932342noreply@blogger.com9tag:blogger.com,1999:blog-1179699229081974369.post-88145222498826187582013-05-27T13:01:00.001-07:002013-06-02T03:05:57.155-07:00How to diagnose OutOfMemoryError in your JVM application?<br />
<div style="text-align: justify;">
<a href="http://images.wikia.com/scpcontainmentbreach/images/6/62/Fire_explosion_rock_7917.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="http://images.wikia.com/scpcontainmentbreach/images/6/62/Fire_explosion_rock_7917.jpg" width="212" /></a><br />
<div>
<h2>
Types of OOME</h2>
So here You are, running Your brand new java application. Or maybe finally the day Your application gained real user traffic came. Your java process started to show in top screen at last and Your server gained some load. Everything was great and suddenly "bang!" - <i>OutOfMemoryError</i>. You restart Your application hoping that was some strange thing and sometimes it helps... for day or so. So You search web to find out how to give java more memory and You double JVM heap size. Sometimes it helps... for two days or so. </div>
<div>
If You see <i>OutOfMemoryError </i>in Your logs, it can mean one of these (<i>percentage of cases based on my real life experience</i>) :</div>
<div>
<ol>
<li>You have memory leak and You need to diagnose it <br />
<span style="font-size: x-small;"><b>(10%</b> of cases)</span></li>
<li>Your application is not using the memory right <br />
<span style="font-size: x-small;">(<b>89.9%</b> of cases)</span></li>
<li>Your application just needs more memory<br />
<span style="font-size: x-small;">(<b>0.1%</b> of cases)</span></li>
</ol>
<div>
Some of You may be surprised that there is something like <b>memory leak</b> in a JAVA program. We must understand that GC surely handles most of the memory allocation issues, but is not a telepath. If You are holding references to objects You are no longer using, and they are still accessible from your applications main reference tree - GC will not free those objects' memory. </div>
</div>
<div>
If Your process downloads hundreds of thousands of records from the database to the memory and than analyses them - Your application is<b> not using the memory righ</b>t. While writing an applications You must predict and control memory usage of the program, so You don't end with great number of objects in Your memory (which can not fit there).<br />
An example of such mistake can be observed here:<br />
<br />
<pre class="brush: java">ResultSet rs = stmt.executeQuery(query);
while (rs.next()) {
executorService.submit(new MyTask(rs.get("id"));
}
executorService.shutdown();
// ... waiting for finish
</pre>
<br />
In most cases in situations like that, task execution takes more time than retrieving next row. While not executed, <span style="font-family: Courier New, Courier, monospace;">MyTask</span> objects wait in <span style="font-family: Courier New, Courier, monospace;">executorService</span><span style="font-family: inherit;"> </span>in-memory queue. Given result set big enough, the memory will end and error will be thrown.<br />
Such errors are nasty, because they appear some time after the application launches. At first, when application has little data, such result sets are little and there is no problem. Application lives, new features are added, things are changed and suddenly <i>OutOfMemoryError </i>appears destroying the application. Such memory usage bug could be with the application from beginning and finding it can cause troubles, as most paniced people will target new functionalities believing that they are responsible for "the new bug".<br />
Sometimes Your application just<b> needs more memory</b>. JAVA is <a href="http://www.ibm.com/developerworks/java/library/j-codetoheap/">kind'a memory consuming</a> and when program runs many threads at the same time, and each of them takes some memory - You need to provide more memory for JVM.<br />
<h2>
Facts about OOME</h2>
<div>
<ul>
<li>Any <i>*Error</i> including <i>OutOfMemoryError </i>destabilizes JVM and it will not perform normally until it will be restarted. </li>
<li>The place <i>OutOfMemoryError </i>is thrown from typically is a random one. It will be thrown first time an object cannot be allocated, not necessarily (and rarely) of the class responsibility for memory problem</li>
<li>You can see <i>OutOfMemoryError </i>is closing by GC behaviour - it will desperately find at least some of the space to use. GC full collect will increase in numbers big time.</li>
</ul>
</div>
<h2>
How to fix OOME?</h2>
<div>
<a href="http://images.wikia.com/x-files/images/a/a9/X-Files_intro.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="236" src="http://images.wikia.com/x-files/images/a/a9/X-Files_intro.jpg" width="320" /></a>When facing <i>OutOfMemoryError </i>Your first task is to diagnose what fills the memory. Your first instinct will be to give java first memory, but it is not likely to help in the long run, as it is rarely the problem with heap size. In most cases it only buys You some time.<br />
Fortunately all the tools necessary to diagnose the problems comes with JVM and You already should have them in Your PATH (if not, You can find them close to other java package binaries) </div>
<div>
Here are the steps You should fallow to diagnose the problem. What is also important: You can run this procedure (and normally will) <b>after</b> <i>OutOfMemoryError </i>occurred. </div>
<div>
<ol>
<li>You must be logged as the owner of the process You want to diagnose.</li>
<li>Get the PID of the process You want to diagnose. You can use Your OS monitoring tool for that, or just use jps<br />
<pre class="brush: java">$ jps -l
32513 sun.tools.jps.Jps
9693 MyApp.jar
</pre>
</li>
<li>Next You need to dump the heap to a file using the pid of our process<br />
<pre class="brush: java">$ jmap -dump:file=dump.map 9693
Dumping heap to /tmp/dump.map ...
Heap dump file created
</pre>
</li>
<li>Such a damp isn't good for reading on Your own. Good someone came with jhat. It is web-based browser through this dump file.<br />
<pre class="brush: java">$ jhat -port 7401 dump.map
Reading from dump.map...
Dump file created Mon Nov 26 10:44:53 CET 2012
Snapshot read, resolving...
Resolving 9942216 objects...
Chasing references, expect 1988 dots....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Eliminating duplicate references....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Snapshot resolved.
Started HTTP server on port 7401
Server is ready.
</pre>
Sometimes the default amount of memory taken by jhat is too little, so You must provide it with more:<br />
<pre class="brush: java">jhat -port 7401 -J-mx4G dump.map
</pre>
</li>
</ol>
</div>
<div>
When You type <b><span style="font-family: Courier New, Courier, monospace;">http://yourhost:7401</span></b> You will see <b>jhat </b>web<b> </b>interface. It takes some getting used to. <a href="http://petermodzelewski.blogspot.com/2013/06/short-jhat-tutorial-diagnosing.html" target="_blank">Here</a> is a tutorial that can help You out. When You know what-is-what in jhat, You must find out:<br />
<br />
<ol>
<li>What objects eat up most of memory </li>
<li>What threads eat up most of the memory</li>
<li>Sometimes - what class loaders eat up most of the memory (when handling application servers to determine which application created the problem). </li>
</ol>
<div>
If diagnosed threads consume predicted amount of memory - then You're lucky and tuning thread number and heap size will solve Your problem (You might also think about getting new server for scaling the workload horizontally)</div>
<div>
In most cases You will need to find the lines of code that produces new instances of problematic class and check if You are not making too many new objects before using them. Maybe some queue grows too quick or list is too big? </div>
<div>
If none of this is true - memory leak occurred and You will need to analyze what is holding references to the objects eating too much memory. That is probably the hardest one to debug. </div>
</div>
<div>
</div>
<br />
<i><span style="font-size: x-small;">Photos courtesy of wikia: <a href="http://scpcontainmentbreach.wikia.com/wiki/File:Fire_explosion_rock_7917.jpg">1</a>, <a href="http://x-files.wikia.com/wiki/File:X-Files_intro.jpg">2</a>.</span></i></div>
<div style="text-align: start;">
</div>
<div style="text-align: start;">
</div>
</div>
KeYeRhttp://www.blogger.com/profile/04045888338509932342noreply@blogger.com0tag:blogger.com,1999:blog-1179699229081974369.post-43291766607112430602013-05-18T12:36:00.002-07:002013-05-18T15:36:22.099-07:00On language popularity in first quarter of 2013<div class="separator" style="clear: both; text-align: justify;">
Recently I found <a href="https://s3.amazonaws.com/aws.drewconway.com/viz/lang_trends/index.html?imm_mid=0a8777&cmp=em-velocity-newsletters-vl-save20-20130516-direct">this great graph</a> showing language popularity basing on stack overflow and github tags:</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMim4-NhFpMZ4C6TsisK4m5I56bFfkyQdJtZnakUa6lQG9_G40X9OKDwZoFHNIHpjVDl7CMe1V5g5YKqDO28hyJAOieDyEdQA7JKCr3dsrdkNFtLylHPH55_x8vJQqmtegqQFuIeUiUVs/s1600/Programming+Language+Trends.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMim4-NhFpMZ4C6TsisK4m5I56bFfkyQdJtZnakUa6lQG9_G40X9OKDwZoFHNIHpjVDl7CMe1V5g5YKqDO28hyJAOieDyEdQA7JKCr3dsrdkNFtLylHPH55_x8vJQqmtegqQFuIeUiUVs/s400/Programming+Language+Trends.png" width="400" /></a></div>
<div style="text-align: justify;">
On the upper right hand side we see the most popular languages: the front line. Most of them are really not a surprise: JAVA, PHP, C/C++, C#, Obj-C, PERL or Ruby. Those are the once that have been in that spotlight for a while now. What's interesting is the strong position of JavaScript (man that language grows) and Python with its second youth sponsored by indie games.</div>
<div style="text-align: justify;">
Not so far away (in compare with older reports) is the second wave of computer languages. One could say that those are the newcomers which are building it's community, but there are some old friends too. The unquestionable leader of the second wave is Scala almost ready to join the mainstream (I've separated it in a one dot set on the graph). The other new popular JVM languages are also there: Clojure and Groovy, although it could have been predicted as those three were gathering bigger and bigger community all the time. It's about time for them to slowly replace JAVA in some applications. </div>
<div style="text-align: justify;">
Interesting that the mentioned old friends: Prolog, Haskel or Lua are there. Those three are really passing the test of time - always in the shadows of mainstream languages but never going down. </div>
<div style="text-align: justify;">
What really got me thinking was how far Rust was from D. Is it that C-family programmers are not so eager to try out new things?<br />
<blockquote class="tr_bq">
<span style="background-color: white;"><span style="color: #333333; font-family: Georgia, Times New Roman, serif;"><span style="font-size: 22px; line-height: 27.984375px; white-space: pre-wrap;">"It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is most adaptable to change."</span></span></span></blockquote>
One thing is certain - new times require new tools, and more and more people realizes that everyday. Mainstream languages will be there with us for long time, but the faster can we adapt to the "second wave" the better our situation on job market will be, and what's more important: the more exiting our everyday work will be. The second wave of languages is on its constant way to join the first way, or even to step into it's place. </div>
KeYeRhttp://www.blogger.com/profile/04045888338509932342noreply@blogger.com6tag:blogger.com,1999:blog-1179699229081974369.post-68551007995506569252013-05-17T11:32:00.003-07:002013-05-27T10:34:03.619-07:00Atmosphere 2013<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZjiAvHRL7yIjG65GPmCoeNGBMpeaaIk4chnyx54nIrxl3uIlCcXbTlHjwyiuR_N-HK32OQyBQTdTNwAO4kHjxmuxysBScLQV4Bwt5lcUWZkRe6o3T0rO6MbfDb5KCpyXDeM0giLGEC0s/s1600/Atmosphere+2013.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: justify;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZjiAvHRL7yIjG65GPmCoeNGBMpeaaIk4chnyx54nIrxl3uIlCcXbTlHjwyiuR_N-HK32OQyBQTdTNwAO4kHjxmuxysBScLQV4Bwt5lcUWZkRe6o3T0rO6MbfDb5KCpyXDeM0giLGEC0s/s1600/Atmosphere+2013.png" /></a></div>
<div style="text-align: justify;">
Earlier this week I had a great pleasure to participate in the first edition of Atmosphere - an e-commerce industry conference. I must admit I went there full of doubts. Can the first edition of such an anonymous event be a success, especially that it was a payed conference? Fortunately it can! When I arrived I saw the whole place buzzing with people talking about all the cool stuff. </div>
<div style="text-align: justify;">
It all started with <a href="https://atmosphere-conference.com/en/lecture/7/">refreshing keynote from Brian McCallister</a>. It was interesting to listen that all startups travel the same evolutionary path. The refreshing part was hearing that "this is OK". That those technical debts we make at the start of a project are caused by good intentions and important reasons. How to mix this general truth with rules of software craftsmanship is yet to be discovered for me. </div>
<blockquote class="tr_bq" style="text-align: justify;">
<span style="background-color: white; color: #333333; font-family: Georgia, 'Times New Roman', serif; font-size: 21.81818199157715px; line-height: 27.99715805053711px; white-space: pre-wrap;">"There are two types of startups. Those successful and a little ashamed of their code and those who're out of business"</span></blockquote>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj26n2wGK2Mi-ej88EsNXARnEP407zTv9HwcNJiI-O-uXJQt0laZh1IJDp5AMhuMDp-v95T5cyuOV2Y7I_406KPgEdFJSLNRFSymAQMY7GM9qWpSGSxeyv8F5lk-4me7SHRB-DHWMZCfIo/s1600/942258_602368383109818_1405336687_n.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em; text-align: justify;"><img border="0" height="212" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj26n2wGK2Mi-ej88EsNXARnEP407zTv9HwcNJiI-O-uXJQt0laZh1IJDp5AMhuMDp-v95T5cyuOV2Y7I_406KPgEdFJSLNRFSymAQMY7GM9qWpSGSxeyv8F5lk-4me7SHRB-DHWMZCfIo/s320/942258_602368383109818_1405336687_n.jpg" width="320" /></a></div>
<div style="text-align: justify;">
Continuing tale of the startups <a href="https://atmosphere-conference.com/en/lecture/11/">Paul Hammond</a> spoke about choices you can make to not run out of money during the first months of the project. That was a really great talk divided into few simple lessons.</div>
<div style="text-align: justify;">
My colleagues from Toruń working on Allegro Charity Platform (running for example <a href="http://wosp.allegro.pl/error/aukcje.wosp.org.pl/index_en.html">wosp.allegro.pl</a>) gave a great <a href="https://atmosphere-conference.com/en/lecture/16/">talk</a> about technology and architecture they've used to handle the traffic of latest WOSP final. </div>
<div style="text-align: justify;">
Talking about handling Allegro traffic <a href="https://atmosphere-conference.com/en/lecture/30/">our </a><a href="https://atmosphere-conference.com/en/lecture/23/">Poznań</a> <a href="https://atmosphere-conference.com/en/lecture/33/">division</a> did a great job sharing some of our experiences in that field fallowed by lecture of both: technologies and methodologies we use to handle such a high traffic with good site responsiveness.</div>
<div style="text-align: justify;">
There were plenty more interesting presentations there but those above are my personal favorites. I'm really looking forward to publication of the videos to watch the ones I missed from other tracks.</div>
<div style="text-align: justify;">
Personally I had an honor to give a <a href="https://atmosphere-conference.com/en/lecture/27/">speech </a>about choosing the programming language for a project covering JAVA, PHP, Erlang and Perl as main characters. I would like to thank all the attendees for being there and I hope you had a good time taking part in this trip through magical lands of programming languages.</div>
<div style="text-align: justify;">
The conference was a blast. On top of great speakers, delicious food and wild party; all of the attendees left with their own raspberry pi as a gadget. I have some plans how to use mine and I will share it with you if I succeed.<br />
I'm looking forward for next years edition. Hope to meet you there too!<br />
<br />
<span style="font-size: x-small;"><i>Photos courtesy of <a href="https://www.facebook.com/AtmosphereConference" target="_blank">Atmosphere Facebook page</a>.</i></span></div>
KeYeRhttp://www.blogger.com/profile/04045888338509932342noreply@blogger.com0Poznań, Poland52.406374 16.92516810000006452.096351 16.279721100000064 52.716397 17.570615100000065tag:blogger.com,1999:blog-1179699229081974369.post-49267685575355286392013-05-16T12:36:00.000-07:002013-05-17T12:45:40.534-07:00Hello World!<div>
It's been a while since I had a blog. Because many people during conferences asked me if I had a blog - I've decided to start one. I believe I have some cool stuff to share. I hope you will find it interesting. Don't hesitate to leave comments to let me know how do you like it. </div>
KeYeRhttp://www.blogger.com/profile/04045888338509932342noreply@blogger.com0