{"action":"created","issue":{"url":"https://api.github.com/repos/xerial/sqlite-jdbc/issues/269","repository_url":"https://api.github.com/repos/xerial/sqlite-jdbc","labels_url":"https://api.github.com/repos/xerial/sqlite-jdbc/issues/269/labels{/name}","comments_url":"https://api.github.com/repos/xerial/sqlite-jdbc/issues/269/comments","events_url":"https://api.github.com/repos/xerial/sqlite-jdbc/issues/269/events","html_url":"https://github.com/xerial/sqlite-jdbc/issues/269","id":256423242,"number":269,"title":"Racy Initialization in SQLiteJDBCLoader","user":{"login":"trohwer","id":6370749,"avatar_url":"https://avatars3.githubusercontent.com/u/6370749?v=4","gravatar_id":"","url":"https://api.github.com/users/trohwer","html_url":"https://github.com/trohwer","followers_url":"https://api.github.com/users/trohwer/followers","following_url":"https://api.github.com/users/trohwer/following{/other_user}","gists_url":"https://api.github.com/users/trohwer/gists{/gist_id}","starred_url":"https://api.github.com/users/trohwer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/trohwer/subscriptions","organizations_url":"https://api.github.com/users/trohwer/orgs","repos_url":"https://api.github.com/users/trohwer/repos","events_url":"https://api.github.com/users/trohwer/events{/privacy}","received_events_url":"https://api.github.com/users/trohwer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-09-09T08:05:31Z","updated_at":"2017-09-09T08:53:02Z","closed_at":null,"author_association":"NONE","body":"If one creates a number of connections concurrently initially, one gets errors like\r\n\r\nException in thread \"Thread-15\" java.lang.UnsatisfiedLinkError: org.sqlite.core.NativeDB._open_utf8([BI)V\r\n        at org.sqlite.core.NativeDB._open_utf8(Native Method)\r\n        at org.sqlite.core.NativeDB._open(NativeDB.java:71)\r\n        at org.sqlite.core.DB.open(DB.java:174)\r\n        at org.sqlite.core.CoreConnection.open(CoreConnection.java:220)\r\n        at org.sqlite.core.CoreConnection.<init>(CoreConnection.java:76)\r\n        at org.sqlite.jdbc3.JDBC3Connection.<init>(JDBC3Connection.java:26)\r\n        at org.sqlite.jdbc4.JDBC4Connection.<init>(JDBC4Connection.java:24)\r\n        at org.sqlite.SQLiteConnection.<init>(SQLiteConnection.java:45)\r\n        at org.sqlite.JDBC.createConnection(JDBC.java:114)\r\n        at org.sqlite.JDBC.connect(JDBC.java:88)\r\n        at java.sql.DriverManager.getConnection(DriverManager.java:664)\r\n        at java.sql.DriverManager.getConnection(DriverManager.java:270)\r\n        at Test3.run(Test3.java:26)\r\n        at java.lang.Thread.run(Thread.java:748)\r\n\r\nThe (- or at least - one) problem is, that the method\r\n\r\ninitialize() in SQLiteJDBCLoader\r\n\r\ncan be entered concurrently (only loadNativeLibrary is covered  by synchonized)\r\nand then one running instance can delete the extracted files of the other running instance.\r\nThe mechanism with the lck files is not reliable; the lock file is created after the dll file.\r\n\r\nHere is a test program:\r\n\r\n`import java.sql.*;\r\n\r\npublic class Test3 implements Runnable {\r\n  static int N= 20;\r\n  static java.util.concurrent.CyclicBarrier cb=\r\n    new java.util.concurrent.CyclicBarrier(N);\r\n  public static void main(String[] args) throws Exception {\r\n    Class.forName(\"org.sqlite.JDBC\");\r\n    for (int i=0; i<N; i++) {\r\n      (new Thread(new Test3(i))).start();\r\n    }\r\n  }\r\n\r\n  int d;\r\n  public Test3(int d) {\r\n    this.d= d;\r\n  }\r\n\r\n  public void run() {\r\n    String url= \"jdbc:sqlite::memory:\";\r\n    try{\r\n      cb.await();\r\n      Thread.sleep(0,100*d);\r\n      DriverManager.getConnection(url).close();\r\n    } catch(Exception e) {\r\n      throw(new RuntimeException(e));\r\n    }\r\n  }\r\n}\r\n`\r\nYou may have to fiddle a bit with the delay to reproduce the bug. Even if it does not trigger,\r\nobserve the multiple dll files created. But this does not only occur in this artifical situation. On a jetty web container it happens almost all the time for me.\r\n\r\nThe whole initialization is somewhat convoluted (Driver/NativeDB/SQLiteJDBCLoader). Probably one should restructure this, but a quick improvement is moving the synchronized to the initialize method\r\nto prevent that race. I will send a pull request.\r\n"},"comment":{"url":"https://api.github.com/repos/xerial/sqlite-jdbc/issues/comments/328264274","html_url":"https://github.com/xerial/sqlite-jdbc/issues/269#issuecomment-328264274","issue_url":"https://api.github.com/repos/xerial/sqlite-jdbc/issues/269","id":328264274,"user":{"login":"trohwer","id":6370749,"avatar_url":"https://avatars3.githubusercontent.com/u/6370749?v=4","gravatar_id":"","url":"https://api.github.com/users/trohwer","html_url":"https://github.com/trohwer","followers_url":"https://api.github.com/users/trohwer/followers","following_url":"https://api.github.com/users/trohwer/following{/other_user}","gists_url":"https://api.github.com/users/trohwer/gists{/gist_id}","starred_url":"https://api.github.com/users/trohwer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/trohwer/subscriptions","organizations_url":"https://api.github.com/users/trohwer/orgs","repos_url":"https://api.github.com/users/trohwer/repos","events_url":"https://api.github.com/users/trohwer/events{/privacy}","received_events_url":"https://api.github.com/users/trohwer/received_events","type":"User","site_admin":false},"created_at":"2017-09-09T08:53:02Z","updated_at":"2017-09-09T08:53:02Z","author_association":"NONE","body":"I can reproduce this on trunk, changing to N=40, 50*d.\r\n\r\nThe actual commit \"Fix issue #197: create lock-file before lib\"\r\nalso does not do, what it claims. The lock file is\r\nstill created after the lib file, it was just moved before writing the content\r\nof the lib file:\r\n\r\nFileOutputStream writer = new FileOutputStream(extractedLibFile);\r\n            if(!extractedLckFile.exists()) {\r\n                new FileOutputStream(extractedLckFile).close();\r\n            }\r\n\r\nEven if we really move the creation of the lock file before the creation of the lib file,\r\nI would avoid assuming, that these file operations are seen in the same order\r\non some other thread. And there are multiple lib/lock files created even during races,\r\nthat don't exhibit the Exception. So I still propose, to move the synchronized.\r\n\r\n"}}
{"actor":{"display_login":"trohwer"}}