{"action":"created","issue":{"url":"https://api.github.com/repos/stumash/CoursePlanner/issues/116","repository_url":"https://api.github.com/repos/stumash/CoursePlanner","labels_url":"https://api.github.com/repos/stumash/CoursePlanner/issues/116/labels{/name}","comments_url":"https://api.github.com/repos/stumash/CoursePlanner/issues/116/comments","events_url":"https://api.github.com/repos/stumash/CoursePlanner/issues/116/events","html_url":"https://github.com/stumash/CoursePlanner/pull/116","id":263757681,"number":116,"title":"Improve backend memory usage","user":{"login":"Davidster","id":2389735,"avatar_url":"https://avatars2.githubusercontent.com/u/2389735?v=4","gravatar_id":"","url":"https://api.github.com/users/Davidster","html_url":"https://github.com/Davidster","followers_url":"https://api.github.com/users/Davidster/followers","following_url":"https://api.github.com/users/Davidster/following{/other_user}","gists_url":"https://api.github.com/users/Davidster/gists{/gist_id}","starred_url":"https://api.github.com/users/Davidster/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Davidster/subscriptions","organizations_url":"https://api.github.com/users/Davidster/orgs","repos_url":"https://api.github.com/users/Davidster/repos","events_url":"https://api.github.com/users/Davidster/events{/privacy}","received_events_url":"https://api.github.com/users/Davidster/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-10-08T22:31:02Z","updated_at":"2017-10-09T05:31:53Z","closed_at":null,"author_association":"COLLABORATOR","pull_request":{"url":"https://api.github.com/repos/stumash/CoursePlanner/pulls/116","html_url":"https://github.com/stumash/CoursePlanner/pull/116","diff_url":"https://github.com/stumash/CoursePlanner/pull/116.diff","patch_url":"https://github.com/stumash/CoursePlanner/pull/116.patch"},"body":"## Summary\r\n\r\n### The problem\r\n\r\nI've been noticing lately that our tomcat server has been shutting down unexpectedly due to an OutOfMemoryError. So I set up memory profiling with a program called yourkit and I noticed that every time a request is made to a servlet which does any db operations (a.k.a `DBServlet`), the memory usage of the application would increase by like 300kb and this memory would not get freed up over time/by garbage collection. I haven't been able to directly reproduce the OutOfMemoryError but I assume it is related to this issue. \r\n\r\n### The solution\r\n\r\nI checked out the type of java objects that were making up the majority of this extra memory usage and it was mostly HashMaps used under the hood by MongoDB (see screenshot below). This led me to re-read the [mongodb docs](http://mongodb.github.io/mongo-java-driver/3.4/driver/getting-started/quick-start/) where I found that the normal usage is to create only one MongoClient object instance to be reused by all servlets in your application. This is different from what we're currently doing which is creating a unique MongoClient object for each servlet. I followed a [guide](https://www.journaldev.com/4011/mongodb-java-servlet-web-application-example-tutorial) on how to share a single reference to a MongoClient object accross all servlets and I made the necessary changes.\r\n\r\nThe version of java we have running on our server is 1.7 so I configured our maven build to use the same version. Also, to be able to use annotations in our backend code I needed to bump up the version of the `javax.servlet` library from `2.5` to `3.1.0`\r\n\r\nAfter this, I found that the problem of memory being used then not freed up on every db request went away. If we ever end up getting an OutOfMemoryError again, I've configured the JVM to do a memory dump when that occurs, in which case we can take a look at what's taking up the most memory in the application.\r\n\r\n![image](https://user-images.githubusercontent.com/2389735/31321538-c28ee646-ac55-11e7-89ed-a52a837a326f.png)\r\n\r\n## Java memory profiling\r\n\r\nIf you want to try out the memory profiling yourself, you can profile our server remotely by installing yourkit and clicking `Connect to remote application` then for the hostname put `conucourseplanner.online`."},"comment":{"url":"https://api.github.com/repos/stumash/CoursePlanner/issues/comments/335069313","html_url":"https://github.com/stumash/CoursePlanner/pull/116#issuecomment-335069313","issue_url":"https://api.github.com/repos/stumash/CoursePlanner/issues/116","id":335069313,"user":{"login":"Davidster","id":2389735,"avatar_url":"https://avatars2.githubusercontent.com/u/2389735?v=4","gravatar_id":"","url":"https://api.github.com/users/Davidster","html_url":"https://github.com/Davidster","followers_url":"https://api.github.com/users/Davidster/followers","following_url":"https://api.github.com/users/Davidster/following{/other_user}","gists_url":"https://api.github.com/users/Davidster/gists{/gist_id}","starred_url":"https://api.github.com/users/Davidster/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Davidster/subscriptions","organizations_url":"https://api.github.com/users/Davidster/orgs","repos_url":"https://api.github.com/users/Davidster/repos","events_url":"https://api.github.com/users/Davidster/events{/privacy}","received_events_url":"https://api.github.com/users/Davidster/received_events","type":"User","site_admin":false},"created_at":"2017-10-09T05:31:53Z","updated_at":"2017-10-09T05:31:53Z","author_association":"COLLABORATOR","body":"tbh I was playing around a lot with the java version on the server when trying to get memory profiling to work. We can probably upgrade to java 8 easily so we should probably do it at some point."}}
{"actor":{"display_login":"Davidster"}}