How to edit outside variables in mapping function
I am trying to find a workaround formy issueand to do so I would like to continue using the mapping function as I have a very large dataset (much bigger than the one in the linked post). I would like to store the multisets in a list of dicts like so:list_data = []
def _multisetify(code_file):
    list_data.append(Multiset(code_file["tokens"]))
    return code_file
data.shuffle() # just to make the tqdm output more accurate
print(data.map(_multisetify, num_proc=32))However this just leaves list_data empty? Does anybody know someway I would be able to do this using the map() function or someway I can do it as quickly as the map function?
2024-08-13T18:53:06Z
Hi again,I suspect that this here and your other issue have the same solution.Issue
2024-08-13T19:44:46Z
It doesn’t raise any error in this one it just doesn’t allow me to edit or create data that can be accessed outside of the function, like if I made a variable in _multisetify() it wouldn’t be able to be accessed outside (as it should), but if I make a variable scoped outside of _multisetify() and edit it inside (with append) then it changes inside the function (I checked), but does not get changed outside _multisetify().
2024-08-13T20:37:10Z
In this example case you would have to also return the list_datareturn code_file, list_dataBut that doesn’t solve your problem.To help debug this issue you might just run a loop over the data.Sorry, hope someone can help,M