Today I made a rather big breakthrough. As you all might know jSunnyreports is extremely slow when it is first started and it uses a huge amount of memory the first time. And the memory issue can be a big problem. In previous versions I introduced the cache mechanism, added a seeding mechanism for creating the database on a fast computer and then moving it to the target computer. And ofcourse the option to manually load each year by hand which is quite error prone.
But what happens when you have 10 inverters and over 6 years of data like I do? Well. I cannot even use jSunnyreports anymore on my core i7 machine with 24gb memory. I simply cannot assign enough memory to my JVM to load all the data and create all the charts at once.
This week I found a solution for this problem. And its quite easy. I have broken up the initial loading process in two distinct steps.
- Load all the data and determine which years are in the set
- Load all the data for a specific year and create charts for that year.
The result is that at any given time only one year of data is retained in memory. The creation of charts is also done on a yearly basis and all the data is saved on a yearly basis as well! That also means that when the process is interrupted it will continue right on the place you want it to.
I have currently assigned 512mb of memory to my JVM I test with. I will do some additional testing to see how low I can go in loading all my data of my 10 inverters.
I don’t know if I will release this version. I had to update the cache structure so everyone who upgrades has to reload its full dataset. And to do that only for this update seems a bit stupid.
More info soon.