{"action":"opened","issue":{"url":"https://api.github.com/repos/JianGoForIt/YellowFin/issues/11","repository_url":"https://api.github.com/repos/JianGoForIt/YellowFin","labels_url":"https://api.github.com/repos/JianGoForIt/YellowFin/issues/11/labels{/name}","comments_url":"https://api.github.com/repos/JianGoForIt/YellowFin/issues/11/comments","events_url":"https://api.github.com/repos/JianGoForIt/YellowFin/issues/11/events","html_url":"https://github.com/JianGoForIt/YellowFin/issues/11","id":241034706,"number":11,"title":"Add YellowFin to tensor2tensor","user":{"login":"ReDeiPirati","id":5721767,"avatar_url":"https://avatars0.githubusercontent.com/u/5721767?v=3","gravatar_id":"","url":"https://api.github.com/users/ReDeiPirati","html_url":"https://github.com/ReDeiPirati","followers_url":"https://api.github.com/users/ReDeiPirati/followers","following_url":"https://api.github.com/users/ReDeiPirati/following{/other_user}","gists_url":"https://api.github.com/users/ReDeiPirati/gists{/gist_id}","starred_url":"https://api.github.com/users/ReDeiPirati/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ReDeiPirati/subscriptions","organizations_url":"https://api.github.com/users/ReDeiPirati/orgs","repos_url":"https://api.github.com/users/ReDeiPirati/repos","events_url":"https://api.github.com/users/ReDeiPirati/events{/privacy}","received_events_url":"https://api.github.com/users/ReDeiPirati/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-07-06T17:47:03Z","updated_at":"2017-07-06T17:47:03Z","closed_at":null,"body":"I am trying to adapt YellowFin to be usable as optimizer in [tensor2tensor](https://github.com/tensorflow/tensor2tensor)(it's use tensorflow>=1.2.0rc1) but unfortunately i cannot debug this error:\r\n\r\n#### Step to reproduce\r\n1. Clone this [repo](https://github.com/ReDeiPirati/tensor2tensor/tree/yellowfin).\r\n2. Launch the `starter.sh` script (inside a Docker container is better).\r\n3. (Optional Docker container command) `nvidia-docker run -it -v $(pwd):/t2t -p 6006:6006 -w /t2t tensorflow/tensorflow:latest-devel-gpu`.\r\n\r\n#### Error\r\n```bash\r\nUsing YellowFin\r\nINFO:tensorflow:Computing gradients for global model_fn.\r\nERROR:tensorflow:==================================\r\nObject was never used (type <class 'tensorflow.python.framework.ops.Operation'>):\r\n<tf.Operation 'training/update_hyper/cond/assert_equal/Assert/Assert' type=Assert>\r\nIf you want to mark it as used call its \"mark_used()\" method.\r\nIt was originally created here:\r\n['File \"/usr/local/bin/t2t-trainer\", line 6, in <module>\\n    exec(compile(open(__file__).read(), __file__, \\'exec\\'))', 'File \"/t2t/tensor2tensor/bin/t2t-trainer\", line 83, in <module>\\n    tf.app.run()', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py\", line 48, in run\\n    _sys.exit(main(_sys.argv[:1] + flags_passthrough))', 'File \"/t2t/tensor2tensor/bin/t2t-trainer\", line 79, in main\\n    schedule=FLAGS.schedule)', 'File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 247, in run\\n    run_locally(exp_fn(output_dir))', 'File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 537, in run_locally\\n    exp.train_and_evaluate()', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 495, in train_and_evaluate\\n    self.train(delay_secs=0)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 275, in train\\n    hooks=self._train_monitors + extra_hooks)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 665, in _call_train\\n    monitors=hooks)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py\", line 289, in new_func\\n    return func(*args, **kwargs)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 455, in fit\\n    loss = self._train_model(input_fn=input_fn, hooks=hooks)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 955, in _train_model\\n    model_fn_ops = self._get_train_ops(features, labels)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 1162, in _get_train_ops\\n    return self._call_model_fn(features, labels, model_fn_lib.ModeKeys.TRAIN)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 1133, in _call_model_fn\\n    model_fn_results = self._model_fn(features, labels, **kwargs)', 'File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 520, in model_fn\\n    colocate_gradients_with_ops=True)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/optimizers.py\", line 293, in optimize_loss\\n    name=\"train\")', 'File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 1154, in apply_gradients\\n    gradients, global_step=global_step, name=name)', 'File \"/t2t/tensor2tensor/utils/yellowfin.py\", line 222, in apply_gradients\\n    update_hyper_op = self.update_hyper_param()', 'File \"/t2t/tensor2tensor/utils/yellowfin.py\", line 190, in update_hyper_param\\n    lambda: self._mu_var) )', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py\", line 289, in new_func\\n    return func(*args, **kwargs)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py\", line 1814, in cond\\n    orig_res_t, res_t = context_t.BuildCondBranch(true_fn)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py\", line 1689, in BuildCondBranch\\n    original_result = fn()', 'File \"/t2t/tensor2tensor/utils/yellowfin.py\", line 189, in <lambda>\\n    self._mu = tf.identity(tf.cond(self._do_tune, lambda: self.get_mu_tensor(),', 'File \"/t2t/tensor2tensor/utils/yellowfin.py\", line 180, in get_mu_tensor\\n    tf.assert_equal(tf.size(root), tf.constant(1) )', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/check_ops.py\", line 318, in assert_equal\\n    return control_flow_ops.Assert(condition, data, summarize=summarize)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/tf_should_use.py\", line 170, in wrapped\\n    return _add_should_use_warning(fn(*args, **kwargs))', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/tf_should_use.py\", line 139, in _add_should_use_warning\\n    wrapped = TFShouldUseWarningWrapper(x)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/tf_should_use.py\", line 96, in __init__\\n    stack = [s.strip() for s in traceback.format_stack()]']\r\n==================================\r\nINFO:tensorflow:Global model_fn finished.\r\nINFO:tensorflow:Create CheckpointSaverHook.\r\n2017-07-06 14:31:31.807218: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.\r\n2017-07-06 14:31:31.807260: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.\r\n2017-07-06 14:31:31.807285: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.\r\n2017-07-06 14:31:31.855132: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\r\n2017-07-06 14:31:31.855471: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties: \r\nname: GeForce GTX 670MX\r\nmajor: 3 minor: 0 memoryClockRate (GHz) 0.601\r\npciBusID 0000:01:00.0\r\nTotal memory: 2.94GiB\r\nFree memory: 2.60GiB\r\n2017-07-06 14:31:31.855541: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 \r\n2017-07-06 14:31:31.855567: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y \r\n2017-07-06 14:31:31.855606: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 670MX, pci bus id: 0000:01:00.0)\r\n2017-07-06 14:31:32.895272: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895276: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895446: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895327: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895466: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895573: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895625: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895675: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895693: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.895545: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.897115: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.901863: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.902270: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.902804: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.903010: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.903597: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.904450: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.904735: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:32.907982: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n2017-07-06 14:31:33.041912: W tensorflow/core/framework/op_kernel.cc:1158] Failed precondition: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\nTraceback (most recent call last):\r\n  File \"/usr/local/bin/t2t-trainer\", line 6, in <module>\r\n    exec(compile(open(__file__).read(), __file__, 'exec'))\r\n  File \"/t2t/tensor2tensor/bin/t2t-trainer\", line 83, in <module>\r\n    tf.app.run()\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py\", line 48, in run\r\n    _sys.exit(main(_sys.argv[:1] + flags_passthrough))\r\n  File \"/t2t/tensor2tensor/bin/t2t-trainer\", line 79, in main\r\n    schedule=FLAGS.schedule)\r\n  File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 247, in run\r\n    run_locally(exp_fn(output_dir))\r\n  File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 537, in run_locally\r\n    exp.train_and_evaluate()\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 495, in train_and_evaluate\r\n    self.train(delay_secs=0)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 275, in train\r\n    hooks=self._train_monitors + extra_hooks)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 665, in _call_train\r\n    monitors=hooks)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py\", line 289, in new_func\r\n    return func(*args, **kwargs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 455, in fit\r\n    loss = self._train_model(input_fn=input_fn, hooks=hooks)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 1003, in _train_model\r\n    config=self._session_config\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 352, in MonitoredTrainingSession\r\n    stop_grace_period_secs=stop_grace_period_secs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 648, in __init__\r\n    stop_grace_period_secs=stop_grace_period_secs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 477, in __init__\r\n    self._sess = _RecoverableSession(self._coordinated_creator)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 822, in __init__\r\n    _WrappedSession.__init__(self, self._create_session())\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 827, in _create_session\r\n    return self._sess_creator.create_session()\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 538, in create_session\r\n    self.tf_sess = self._session_creator.create_session()\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 412, in create_session\r\n    init_fn=self._scaffold.init_fn)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/session_manager.py\", line 279, in prepare_session\r\n    sess.run(init_op, feed_dict=init_feed_dict)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py\", line 789, in run\r\n    run_metadata_ptr)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py\", line 997, in _run\r\n    feed_dict_string, options, run_metadata)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py\", line 1132, in _do_run\r\n    target_list, options, run_metadata)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py\", line 1152, in _do_call\r\n    raise type(e)(node_def, op, message)\r\ntensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n\r\nCaused by op u'global_step/read', defined at:\r\n  File \"/usr/local/bin/t2t-trainer\", line 6, in <module>\r\n    exec(compile(open(__file__).read(), __file__, 'exec'))\r\n  File \"/t2t/tensor2tensor/bin/t2t-trainer\", line 83, in <module>\r\n    tf.app.run()\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py\", line 48, in run\r\n    _sys.exit(main(_sys.argv[:1] + flags_passthrough))\r\n  File \"/t2t/tensor2tensor/bin/t2t-trainer\", line 79, in main\r\n    schedule=FLAGS.schedule)\r\n  File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 247, in run\r\n    run_locally(exp_fn(output_dir))\r\n  File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 537, in run_locally\r\n    exp.train_and_evaluate()\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 495, in train_and_evaluate\r\n    self.train(delay_secs=0)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 275, in train\r\n    hooks=self._train_monitors + extra_hooks)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 665, in _call_train\r\n    monitors=hooks)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py\", line 289, in new_func\r\n    return func(*args, **kwargs)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 455, in fit\r\n    loss = self._train_model(input_fn=input_fn, hooks=hooks)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 952, in _train_model\r\n    global_step = contrib_framework.create_global_step(g)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/variables.py\", line 133, in create_global_step\r\n    return training_util.create_global_step(graph)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/training_util.py\", line 119, in create_global_step\r\n    collections=[ops.GraphKeys.GLOBAL_VARIABLES, ops.GraphKeys.GLOBAL_STEP])\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py\", line 1065, in get_variable\r\n    use_resource=use_resource, custom_getter=custom_getter)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py\", line 962, in get_variable\r\n    use_resource=use_resource, custom_getter=custom_getter)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py\", line 367, in get_variable\r\n    validate_shape=validate_shape, use_resource=use_resource)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py\", line 352, in _true_getter\r\n    use_resource=use_resource)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py\", line 725, in _get_single_variable\r\n    validate_shape=validate_shape)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variables.py\", line 200, in __init__\r\n    expected_shape=expected_shape)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variables.py\", line 319, in _init_from_args\r\n    self._snapshot = array_ops.identity(self._variable, name=\"read\")\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py\", line 1303, in identity\r\n    result = _op_def_lib.apply_op(\"Identity\", input=input, name=name)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py\", line 767, in apply_op\r\n    op_def=op_def)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py\", line 2506, in create_op\r\n    original_op=self._default_original_op, op_def=op_def)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py\", line 1269, in __init__\r\n    self._traceback = _extract_stack()\r\n\r\nFailedPreconditionError (see above for traceback): Attempting to use uninitialized value global_step\r\n\t [[Node: global_step/read = Identity[T=DT_INT64, _class=[\"loc:@global_step\"], _device=\"/job:localhost/replica:0/task:0/cpu:0\"](global_step)]]\r\n\r\nERROR:tensorflow:==================================\r\nObject was never used (type <class 'tensorflow.python.framework.ops.Tensor'>):\r\n<tf.Tensor 'report_uninitialized_variables_1/boolean_mask/Gather:0' shape=(?,) dtype=string>\r\nIf you want to mark it as used call its \"mark_used()\" method.\r\nIt was originally created here:\r\n['File \"/usr/local/bin/t2t-trainer\", line 6, in <module>\\n    exec(compile(open(__file__).read(), __file__, \\'exec\\'))', 'File \"/t2t/tensor2tensor/bin/t2t-trainer\", line 83, in <module>\\n    tf.app.run()', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py\", line 48, in run\\n    _sys.exit(main(_sys.argv[:1] + flags_passthrough))', 'File \"/t2t/tensor2tensor/bin/t2t-trainer\", line 79, in main\\n    schedule=FLAGS.schedule)', 'File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 247, in run\\n    run_locally(exp_fn(output_dir))', 'File \"/t2t/tensor2tensor/utils/trainer_utils.py\", line 537, in run_locally\\n    exp.train_and_evaluate()', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 495, in train_and_evaluate\\n    self.train(delay_secs=0)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 275, in train\\n    hooks=self._train_monitors + extra_hooks)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/experiment.py\", line 665, in _call_train\\n    monitors=hooks)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py\", line 289, in new_func\\n    return func(*args, **kwargs)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 455, in fit\\n    loss = self._train_model(input_fn=input_fn, hooks=hooks)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py\", line 1003, in _train_model\\n    config=self._session_config', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 352, in MonitoredTrainingSession\\n    stop_grace_period_secs=stop_grace_period_secs)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 648, in __init__\\n    stop_grace_period_secs=stop_grace_period_secs)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 477, in __init__\\n    self._sess = _RecoverableSession(self._coordinated_creator)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 822, in __init__\\n    _WrappedSession.__init__(self, self._create_session())', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 827, in _create_session\\n    return self._sess_creator.create_session()', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 538, in create_session\\n    self.tf_sess = self._session_creator.create_session()', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 403, in create_session\\n    self._scaffold.finalize()', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 192, in finalize\\n    default_ready_for_local_init_op)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 254, in get_or_default\\n    op = default_constructor()', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/monitored_session.py\", line 189, in default_ready_for_local_init_op\\n    variables.global_variables())', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/tf_should_use.py\", line 170, in wrapped\\n    return _add_should_use_warning(fn(*args, **kwargs))', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/tf_should_use.py\", line 139, in _add_should_use_warning\\n    wrapped = TFShouldUseWarningWrapper(x)', 'File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/tf_should_use.py\", line 96, in __init__\\n    stack = [s.strip() for s in traceback.format_stack()]']\r\n==================================\r\n```\r\n\r\nIf you do not want to help or contribute, please close the issue and forgive me.\r\nOtherwise, i will appreciate any help :)\r\n\r\nI've also tried to write YellowFin as an tf.train.Optimizer, but going at C++ level seems to be out of my skills at the moment... "}}
{"actor":{"display_login":"ReDeiPirati"}}