{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/13463","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/13463/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/13463/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/13463/events","html_url":"https://github.com/tensorflow/tensorflow/issues/13463","id":262402818,"number":13463,"title":"DataLossError (see above for traceback): corrupted record at 12","user":{"login":"huangrandong","id":32484109,"avatar_url":"https://avatars0.githubusercontent.com/u/32484109?v=4","gravatar_id":"","url":"https://api.github.com/users/huangrandong","html_url":"https://github.com/huangrandong","followers_url":"https://api.github.com/users/huangrandong/followers","following_url":"https://api.github.com/users/huangrandong/following{/other_user}","gists_url":"https://api.github.com/users/huangrandong/gists{/gist_id}","starred_url":"https://api.github.com/users/huangrandong/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huangrandong/subscriptions","organizations_url":"https://api.github.com/users/huangrandong/orgs","repos_url":"https://api.github.com/users/huangrandong/repos","events_url":"https://api.github.com/users/huangrandong/events{/privacy}","received_events_url":"https://api.github.com/users/huangrandong/received_events","type":"User","site_admin":false},"labels":[{"id":386191887,"url":"https://api.github.com/repos/tensorflow/tensorflow/labels/stat:awaiting%20response","name":"stat:awaiting response","color":"5319e7","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2017-10-03T12:08:07Z","updated_at":"2017-10-12T11:39:39Z","closed_at":null,"author_association":"NONE","body":"I have a big problem, I use the  tfrecord file to import data for my tensorflow program. But, when the program run a period of time， it displays the DataLossError:\r\n------------------------\r\n\r\n### System information\r\nOS Platform and Distribution : Linux Ubuntu 14.04\r\nTensorFlow installed from : Anaconda\r\nTensorFlow version : 1.3.0\r\nPython version:  2.7.13\r\nCUDA/cuDNN version: 8.0 / 6.0  \r\nGPU model and memory: Pascal TITAN X\r\n\r\n### Describe the problem\r\n 2017-10-03 19:45:43.854601: W tensorflow/core/framework/op_kernel.cc:1192] Data loss: corrupted record at 12\r\nTraceback (most recent call last):\r\n  File \"east_quad_train_backup.py\", line 416, in <module>\r\n    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py\", line 48, in run\r\n    _sys.exit(main(_sys.argv[:1] + flags_passthrough))\r\n  File \"east_quad_train_backup.py\", line 330, in main\r\n    Training()\r\n  File \"east_quad_train_backup.py\", line 312, in Training\r\n    feed_dict={learning_rate: lr})\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py\", line 895, in run\r\n    run_metadata_ptr)\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py\", line 1124, in _run\r\n    feed_dict_tensor, options, run_metadata)\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py\", line 1321, in _do_run\r\n    options, run_metadata)\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py\", line 1340, in _do_call\r\n    raise type(e)(node_def, op, message)\r\ntensorflow.python.framework.errors_impl.DataLossError: corrupted record at 12\r\n\t [[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[?,512,512,3], [?,128,128,9]], output_types=[DT_UINT8, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Iterator)]]\r\n\t [[Node: gradients/Tile_grad/Shape/_23 = _HostRecv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/gpu:0\", send_device=\"/job:localhost/replica:0/task:0/cpu:0\", send_device_incarnation=1, tensor_name=\"edge_442_gradients/Tile_grad/Shape\", tensor_type=DT_INT32, _device=\"/job:localhost/replica:0/task:0/gpu:0\"]()]]\r\n\r\nCaused by op u'IteratorGetNext', defined at:\r\n  File \"east_quad_train_backup.py\", line 416, in <module>\r\n    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py\", line 48, in run\r\n    _sys.exit(main(_sys.argv[:1] + flags_passthrough))\r\n  File \"east_quad_train_backup.py\", line 330, in main\r\n    Training()\r\n  File \"east_quad_train_backup.py\", line 251, in Training\r\n    batch_image, batch_label = iterator.get_next()\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/data/python/ops/dataset_ops.py\", line 304, in get_next\r\n    name=name))\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_dataset_ops.py\", line 379, in iterator_get_next\r\n    output_shapes=output_shapes, name=name)\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py\", line 767, in apply_op\r\n    op_def=op_def)\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py\", line 2630, in create_op\r\n    original_op=self._default_original_op, op_def=op_def)\r\n  File \"/home/t/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py\", line 1204, in __init__\r\n    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access\r\n\r\nDataLossError (see above for traceback): corrupted record at 12\r\n\t [[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[?,512,512,3], [?,128,128,9]], output_types=[DT_UINT8, DT_FLOAT], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](Iterator)]]\r\n\t [[Node: gradients/Tile_grad/Shape/_23 = _HostRecv[client_terminated=false, recv_device=\"/job:localhost/replica:0/task:0/gpu:0\", send_device=\"/job:localhost/replica:0/task:0/cpu:0\", send_device_incarnation=1, tensor_name=\"edge_442_gradients/Tile_grad/Shape\", tensor_type=DT_INT32, _device=\"/job:localhost/replica:0/task:0/gpu:0\"]()]]\r\n\r\n\r\nThanks anyone to answer this question."},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/336102060","html_url":"https://github.com/tensorflow/tensorflow/issues/13463#issuecomment-336102060","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/13463","id":336102060,"user":{"login":"huangrandong","id":32484109,"avatar_url":"https://avatars0.githubusercontent.com/u/32484109?v=4","gravatar_id":"","url":"https://api.github.com/users/huangrandong","html_url":"https://github.com/huangrandong","followers_url":"https://api.github.com/users/huangrandong/followers","following_url":"https://api.github.com/users/huangrandong/following{/other_user}","gists_url":"https://api.github.com/users/huangrandong/gists{/gist_id}","starred_url":"https://api.github.com/users/huangrandong/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huangrandong/subscriptions","organizations_url":"https://api.github.com/users/huangrandong/orgs","repos_url":"https://api.github.com/users/huangrandong/repos","events_url":"https://api.github.com/users/huangrandong/events{/privacy}","received_events_url":"https://api.github.com/users/huangrandong/received_events","type":"User","site_admin":false},"created_at":"2017-10-12T11:39:39Z","updated_at":"2017-10-12T11:39:39Z","author_association":"NONE","body":"@reedwm  my code is used to put the numpy array into  the TFrecord file and read the it from the same file ,this is my code:\r\n# create the TFrecord file function:\r\nimg_tfrecord_name = image_base_name + \".tfrecord\"\r\nwriter = tf.python_io.TFRecordWriter(new_label_path + img_tfrecord_name)\r\nlabel_concate = np.concatenate((score_map, x1_offset, y1_offset,\r\n                                                x2_offset, y2_offset, x3_offset,\r\n                                                y3_offset, x4_offset, y4_offset), axis = -1)\r\norg_train_image = cv2.imread(org_train_images_path + img_name)\r\norg_train_image_resize = cv2.resize(org_train_image,\r\n                                        (input_image_size, input_image_size))\r\nassert org_train_image_resize.shape == (512,512,3)\r\norg_train_image_resize = org_train_image_resize.astype(np.uint8)\r\norg_train_image_resize_raw = org_train_image_resize.tostring()\r\nlabel_concate = label_concate.astype(np.float32)\r\nlabel_concate_raw = label_concate.tostring()\r\nexample = tf.train.Example(\r\n                     features = tf.train.Features(\r\n                     feature = {'image':tf.train.Feature(bytes_list = \r\n                                       tf.train.BytesList(value[org_train_image_resize_raw])),\r\n                                      'label':tf.train.Feature(bytes_list = tf.train.BytesList(value=[label_concate_raw]))}))\r\nserialized = example.SerializeToString()\r\nwriter.write(serialized)\r\nprint 'writer  ',img_name,'  DOWN!'\r\nwriter.close()\r\n\r\n# read the TFrecord file ：\r\ndef _parse_function_for_train(example_proto):\r\n       features = {'image': tf.FixedLenFeature((), tf.string, default_value=\"\"),\r\n                          'label': tf.FixedLenFeature((), tf.string, default_value=\"\")}\r\n        parsed_features = tf.parse_single_example(example_proto, features)\r\n        image_raw_out = parsed_features['image']\r\n        label_raw_out = parsed_features['label']\r\n        image_out = tf.decode_raw(image_raw_out, tf.uint8)\r\n        label_out = tf.decode_raw(label_raw_out, tf.float32)\r\n        image_out = tf.reshape(image_out, [512, 512, 3])\r\n        label_out = tf.reshape(label_out, [128,128,9])\r\n        return image_out, label_out\r\n\r\ndef CreateTrainDataset():\r\n      train_image_label_tfrecord_list = [\"t1.tfrecord\", \"t2.tfrecord\",......]\r\n      train_dataset = tf.contrib.data.TFRecordDataset(train_image_label_tfrecord_list)\r\n      train_dataset = train_dataset.map(_parse_function_for_train)\r\n      batched_train_dataset = train_dataset.batch(512)\r\n      return batched_train_dataset\r\n\r\nbatched_train_dataset = CreateTrainDataset()\r\niterator = batched_train_dataset.make_initializable_iterator()\r\nbatch_image, batch_label = iterator.get_next()\r\nwith tf.Session() as sess:\r\n          sess.run(iterator.initializer)\r\nWhen the above code run some iterations, the DataLossError will com out\r\n\r\n"}}
{"actor":{"display_login":"huangrandong"}}