{"action":"created","issue":{"url":"https://api.github.com/repos/dmwm/WMCore/issues/8258","repository_url":"https://api.github.com/repos/dmwm/WMCore","labels_url":"https://api.github.com/repos/dmwm/WMCore/issues/8258/labels{/name}","comments_url":"https://api.github.com/repos/dmwm/WMCore/issues/8258/comments","events_url":"https://api.github.com/repos/dmwm/WMCore/issues/8258/events","html_url":"https://github.com/dmwm/WMCore/issues/8258","id":266462395,"number":8258,"title":"Merge jobs failing with 70452: NoRunLumiInformation","user":{"login":"amaltaro","id":1798520,"avatar_url":"https://avatars0.githubusercontent.com/u/1798520?v=4","gravatar_id":"","url":"https://api.github.com/users/amaltaro","html_url":"https://github.com/amaltaro","followers_url":"https://api.github.com/users/amaltaro/followers","following_url":"https://api.github.com/users/amaltaro/following{/other_user}","gists_url":"https://api.github.com/users/amaltaro/gists{/gist_id}","starred_url":"https://api.github.com/users/amaltaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amaltaro/subscriptions","organizations_url":"https://api.github.com/users/amaltaro/orgs","repos_url":"https://api.github.com/users/amaltaro/repos","events_url":"https://api.github.com/users/amaltaro/events{/privacy}","received_events_url":"https://api.github.com/users/amaltaro/received_events","type":"User","site_admin":false},"labels":[{"id":31815328,"url":"https://api.github.com/repos/dmwm/WMCore/labels/High%20Priority","name":"High Priority","color":"e10c02","default":false}],"state":"open","locked":false,"assignee":{"login":"amaltaro","id":1798520,"avatar_url":"https://avatars0.githubusercontent.com/u/1798520?v=4","gravatar_id":"","url":"https://api.github.com/users/amaltaro","html_url":"https://github.com/amaltaro","followers_url":"https://api.github.com/users/amaltaro/followers","following_url":"https://api.github.com/users/amaltaro/following{/other_user}","gists_url":"https://api.github.com/users/amaltaro/gists{/gist_id}","starred_url":"https://api.github.com/users/amaltaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amaltaro/subscriptions","organizations_url":"https://api.github.com/users/amaltaro/orgs","repos_url":"https://api.github.com/users/amaltaro/repos","events_url":"https://api.github.com/users/amaltaro/events{/privacy}","received_events_url":"https://api.github.com/users/amaltaro/received_events","type":"User","site_admin":false},"assignees":[{"login":"amaltaro","id":1798520,"avatar_url":"https://avatars0.githubusercontent.com/u/1798520?v=4","gravatar_id":"","url":"https://api.github.com/users/amaltaro","html_url":"https://github.com/amaltaro","followers_url":"https://api.github.com/users/amaltaro/followers","following_url":"https://api.github.com/users/amaltaro/following{/other_user}","gists_url":"https://api.github.com/users/amaltaro/gists{/gist_id}","starred_url":"https://api.github.com/users/amaltaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amaltaro/subscriptions","organizations_url":"https://api.github.com/users/amaltaro/orgs","repos_url":"https://api.github.com/users/amaltaro/repos","events_url":"https://api.github.com/users/amaltaro/events{/privacy}","received_events_url":"https://api.github.com/users/amaltaro/received_events","type":"User","site_admin":false}],"milestone":null,"comments":12,"created_at":"2017-10-18T11:43:38Z","updated_at":"2017-10-24T09:37:38Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"To be investigated once https://github.com/dmwm/WMCore/issues/8220 is sorted out.\r\nFrom JIRA ticket:\r\nhttps://its.cern.ch/jira/browse/CMSCOMPPR-1552\r\n\r\nand from wmstats:\r\n```\r\n    Job Name: b7ae480e-b03b-11e7-bce9-02163e01877e-0\r\n    WMBS job id: 684921\r\n    Workflow: fabozzi_Run2017C-v1-DoubleMuonLowMass-12Sep2017_9211_170922_194400_3624\r\n    Task: /fabozzi_Run2017C-v1-DoubleMuonLowMass-12Sep2017_9211_170922_194400_3624/DataProcessing/DataProcessingMergeDQMoutput\r\n    Status: jobfailed\r\n    Input dataset: /DoubleMuonLowMass/Run2017C-v1/RAW\r\n    Site: T2_UK_London_Brunel\r\n    Agent: vocms0250.cern.ch\r\n    ACDC URL: https://cmsweb.cern.ch/couchdb/acdcserver\r\n    State Transition\r\n    Exit code: 70452\r\n    Retry count: 4\r\n    Errors:\r\n        logArch1\r\n        cmsRun1\r\n            NoRunLumiInformation (Exit Code: 70452)\r\n\r\n                No run/lumi information in file (WMAgent)., file was /store/data/Run2017C/DoubleMuonLowMass/DQMIO/12Sep2017-v1/00000/98FA9531-01B1-E711-85F5-0025904C7E04.root\r\n\r\n        stageOut1\r\n            ReportManipulatingError (Exit Code: 99999)\r\n\r\n                Could not find report file for step stageOut1!\r\n\r\n    Input files:\r\n    Lumis:\r\n    Output:\r\n        output\r\n            lfn: /store/data/Run2017C/DoubleMuonLowMass/DQMIO/12Sep2017-v1/00000/98FA9531-01B1-E711-85F5-0025904C7E04.root\r\n            location: null\r\n            checksums: adler32: 749f8434\r\n            size: 17030\r\n        logArchive\r\n            lfn: /store/unmerged/logs/prod/2017/10/14/fabozzi_Run2017C-v1-DoubleMuonLowMass-12Sep2017_9211_170922_194400_3624/DataProcessing/DataProcessingMergeDQMoutput/0000/4/b7ae480e-b03b-11e7-bce9-02163e01877e-0-4-logArchive.tar.gz\r\n            location: null\r\n            pfn:\r\n            checksums: adler32: ee97d292\r\n            size: 0\r\n    log archive:\r\n```"},"comment":{"url":"https://api.github.com/repos/dmwm/WMCore/issues/comments/338933258","html_url":"https://github.com/dmwm/WMCore/issues/8258#issuecomment-338933258","issue_url":"https://api.github.com/repos/dmwm/WMCore/issues/8258","id":338933258,"user":{"login":"amaltaro","id":1798520,"avatar_url":"https://avatars0.githubusercontent.com/u/1798520?v=4","gravatar_id":"","url":"https://api.github.com/users/amaltaro","html_url":"https://github.com/amaltaro","followers_url":"https://api.github.com/users/amaltaro/followers","following_url":"https://api.github.com/users/amaltaro/following{/other_user}","gists_url":"https://api.github.com/users/amaltaro/gists{/gist_id}","starred_url":"https://api.github.com/users/amaltaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amaltaro/subscriptions","organizations_url":"https://api.github.com/users/amaltaro/orgs","repos_url":"https://api.github.com/users/amaltaro/repos","events_url":"https://api.github.com/users/amaltaro/events{/privacy}","received_events_url":"https://api.github.com/users/amaltaro/received_events","type":"User","site_admin":false},"created_at":"2017-10-24T09:37:38Z","updated_at":"2017-10-24T09:37:38Z","author_association":"CONTRIBUTOR","body":"About the short-term fix, we need to (of course, with components down):\r\n1. remove rows for `('T3_UK_London_QMUL', 'T3_UK_ScotGrid_GLA')` from wmbs_location\r\n2. same for wmbs_location_pnns\r\n3. same for wmbs_file_location, in the end files with these location have also other valid locations\r\n\r\nthen on the DBSBuffer tables:\r\n4. remove same rows from dbsbuffer_location\r\n5. remove rows from dbsbuffer_file_location\r\n6. and remove rows from dbsbuffer_block (it appears to be none atm)\r\n\r\n@ticoann @hufnagel am I missing something? \r\nA conservative approach would be to live with this problem until we deploy new agents (and don't add T3s to resource control). Let me know please."}}
{"actor":{"display_login":"amaltaro"}}