{"action":"opened","issue":{"url":"https://api.github.com/repos/trailofbits/mcsema/issues/108","repository_url":"https://api.github.com/repos/trailofbits/mcsema","labels_url":"https://api.github.com/repos/trailofbits/mcsema/issues/108/labels{/name}","comments_url":"https://api.github.com/repos/trailofbits/mcsema/issues/108/comments","events_url":"https://api.github.com/repos/trailofbits/mcsema/issues/108/events","html_url":"https://github.com/trailofbits/mcsema/issues/108","id":208533402,"number":108,"title":"Some data refs not recognized because they aren't item heads","user":{"login":"pgoodman","id":43664,"avatar_url":"https://avatars.githubusercontent.com/u/43664?v=3","gravatar_id":"","url":"https://api.github.com/users/pgoodman","html_url":"https://github.com/pgoodman","followers_url":"https://api.github.com/users/pgoodman/followers","following_url":"https://api.github.com/users/pgoodman/following{/other_user}","gists_url":"https://api.github.com/users/pgoodman/gists{/gist_id}","starred_url":"https://api.github.com/users/pgoodman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pgoodman/subscriptions","organizations_url":"https://api.github.com/users/pgoodman/orgs","repos_url":"https://api.github.com/users/pgoodman/repos","events_url":"https://api.github.com/users/pgoodman/events{/privacy}","received_events_url":"https://api.github.com/users/pgoodman/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-02-17T19:26:55Z","updated_at":"2017-02-17T19:26:55Z","closed_at":null,"body":"One of the heuristics for recognizing a data ref is by taking something that looks like an address into the data section, and asking if `idc.ItemHead(addr) == addr`. This particular condition is insufficient in a few related cases.\r\n\r\nI'm observing this issue in `/bin/ls`, specifically when I run `/bin/ls --width=100`. The function `getopt_long` returns the wrong results, and the reason why becomes obvious when you print out the lifted  options array passed to the function:\r\n\r\n```\r\n$6 = {name = 0x585cc8 <data_413ca0+11400> \"all\", has_arg = 0, flag = 0x0, val = 97}\r\n...\r\n$18 = {name = 0x585cb9 <data_413ca0+11385> \"reverse\", has_arg = 0, flag = 0x0, val = 114}\r\n$19 = {name = 0x41666b <sub_40F4A0+64411> \"\\003\", has_arg = 0, flag = 0x0, val = 115}\r\n$20 = {name = 0x41664e <sub_40F4A0+64382> \"\\003\", has_arg = 1, flag = 0x0, val = 119}\r\n```\r\n\r\nThis should look more like that in the native one:\r\n```\r\n$3 = {name = 0x416928 \"all\", has_arg = 0, flag = 0x0, val = 97}\r\n...\r\n$15 = {name = 0x416919 \"reverse\", has_arg = 0, flag = 0x0, val = 114}\r\n$16 = {name = 0x41666b \"size\", has_arg = 0, flag = 0x0, val = 115}\r\n$17 = {name = 0x41664e \"width\", has_arg = 1, flag = 0x0, val = 119}\r\n$18 = {name = 0x416921 \"almost-all\", has_arg = 0, flag = 0x0, val = 65}\r\n```\r\n\r\nThere are two interesting things to see here. I have temporarily \"solved\" the first issue, which is an `ItemHead` check in the `isSaneReference` function in `get_cfg.py`. The problem was with the first argument, `\"all\"`. The thing to notice is that the compiler performed string deduplication, and so the `\"all\"` is actually a substring of a later option string `\"almost-all\"`. This failure of the `ItemHead` check is not too suprising.\r\n\r\nI'm not so sure about the other failures, related to `\"size\"` and `\"width\"`. It seems like some of these internal elements of the array just aren't recognized as such, or that the strings in them fail the `ItemHead` test. In the lift log I see something like this: \r\n```\r\nTesting address: 414f80... \r\nData Range: 413ca0 <= 41664e < 41a654\r\nData Range: 41664e - 41664f\r\nnot code or data ref\r\n```\r\nWhich corresponds to a failure of the `idc.ItemHead` condition in:\r\n```python\r\n            # check for unmakred references\r\n            # that point to the beginning of an item\r\n            if isInData(dword, dword+1) and idc.ItemHead(dword) == dword:\r\n                ...\r\n```\r\n\r\nI'm not yet certain how I want to handle this, as a cure may be worse than the disease :-("}}
{"actor":{"display_login":"pgoodman"}}