You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Need to sanitize model output with postprocessing function to ensure it is a numpy array (assuming this is the expected type). Additionally need some more verbose type information 😅
Steps To Reproduce
Run MITM objdet with export_batches on
Additional Information
$ python -m armory run ./eval7/mitm_eval7_objdet/advpatch_rgb.json --num-eval-batches=1
2023-08-17 17:51:58 0s INFO armory.eval.evaluator:init:30 Constructing Evaluator Object
2023-08-17 17:51:58 0s INFO armory.eval.evaluator:init:61 Retrieving Environment Variables
2023-08-17 17:51:58 0s SUCCESS armory.docker.images:is_image_local:133 found docker image mitm_eval7_objdet:1.1
2023-08-17 17:51:59 2s INFO armory.docker.management:init:62 ARMORY Instance 7d8b44bd5d1b created.
2023-08-17 17:51:59 2s INFO armory.eval.evaluator:_run_config:287 Running evaluation script
2023-08-17 17:52:00 0s INFO armory.delayed_imports::7 Importing and configuring torch, tensorflow, and art, if available. This may take some time.
2023-08-17 17:52:04 4s INFO armory.delayed_imports::30 Setting tf.config.experimental.set_memory_growth to True on all GPUs
2023-08-17 17:52:04 4s INFO art.config:set_data_path:55 set ART_DATA_PATH to /tmp/.art/data
2023-08-17 17:52:05 5s INFO art.config:set_data_path:55 set ART_DATA_PATH to /armory/saved_models/art
2023-08-17 17:52:07 7s INFO matplotlib.font_manager:_load_fontmanager:1624 generated new fontManager
2023-08-17 17:52:11 10s INFO armory.data.utils:maybe_download_weights_from_s3:47 Using available MITM-objdet_rgb.pt in Armory saved_model_dir
2023-08-17 17:52:16 16s INFO armory.scenarios.scenario:load_dataset:257 Loading test dataset carla_over_obj_det_dev...
2023-08-17 17:52:16 16s WARNING tensorflow.python.platform.tf_logging:warning:173 From /opt/conda/lib/python3.9/site-packages/tensorflow/python/autograph/pyct/static_analysis/liveness.py:83: Analyzer.lamba_check (from tensorflow.python.autograph.pyct.static_analysis.liveness) is deprecated and will be removed after 2023-09-23.
Instructions for updating:
Lambda fuctions will be no more assumed to be used in the statement where they are used, or at least in the same block. tensorflow/tensorflow#56089
2023-08-17 17:52:16 16s INFO armory.scenarios.scenario:evaluate_all:335 Running inference on benign and adversarial examples
Evaluation: 0% 0/1 [00:00<?, ?it/s]
Evaluation: 0% 0/1 [00:06<?, ?it/s]
2023-08-17 17:52:22 22s ERROR armory.scenarios.scenario:evaluate:452 Encountered error during scenario evaluation.
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
│ │ └ {'name': 'main', 'doc': '\nMain script for running scenarios. Users will run a scenario by calling:\n armory r...
│ └ <code object at 0x7fe22f6bb240, file "/workspace/armory/scenarios/main.py", line 1>
└ <function _run_code at 0x7fe31a85f9d0>
File "/opt/conda/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
│ └ {'name': 'main', 'doc': '\nMain script for running scenarios. Users will run a scenario by calling:\n armory r...
└ <code object at 0x7fe22f6bb240, file "/workspace/armory/scenarios/main.py", line 1>
File "/workspace/armory/scenarios/main.py", line 229, in
run_config(
└ <function run_config at 0x7fe22e52e1f0>
File "/workspace/armory/scenarios/main.py", line 145, in run_config
scenario.evaluate()
│ └ <function Scenario.evaluate at 0x7fe202cdf670>
└ <armory.scenarios.carla_object_detection.CarlaObjectDetectionTask object at 0x7fe22f385730>
File "/workspace/armory/scenarios/scenario.py", line 443, in evaluate
self._evaluate()
│ └ <function Scenario._evaluate at 0x7fe202cdf5e0>
└ <armory.scenarios.carla_object_detection.CarlaObjectDetectionTask object at 0x7fe22f385730>
File "/workspace/armory/scenarios/scenario.py", line 433, in _evaluate
self.evaluate_all()
│ └ <function Scenario.evaluate_all at 0x7fe202cdf1f0>
└ <armory.scenarios.carla_object_detection.CarlaObjectDetectionTask object at 0x7fe22f385730>
File "/workspace/armory/scenarios/scenario.py", line 338, in evaluate_all
self.evaluate_current()
│ └ <function Scenario.evaluate_current at 0x7fe202cdf4c0>
└ <armory.scenarios.carla_object_detection.CarlaObjectDetectionTask object at 0x7fe22f385730>
File "/workspace/armory/scenarios/scenario.py", line 417, in evaluate_current
self.run_benign()
│ └ <function Scenario.run_benign at 0x7fe202cdf3a0>
└ <armory.scenarios.carla_object_detection.CarlaObjectDetectionTask object at 0x7fe22f385730>
File "/workspace/armory/scenarios/scenario.py", line 363, in run_benign
self.probe.update(y_pred=y_pred)
│ │ │ └ [{'boxes': tensor([[ 417.6945, 667.3348, 547.8636, 767.3787],
│ │ │ [ 446.4780, 769.8671, 609.9299, 900.4496],
│ │ │ ...
│ │ └ <function Probe.update at 0x7fe202e2b700>
│ └ <armory.instrument.instrument.Probe object at 0x7fe22e4ddf40>
└ <armory.scenarios.carla_object_detection.CarlaObjectDetectionTask object at 0x7
fe22f385730>
File "/workspace/armory/instrument/instrument.py", line 118, in update
self.sink.update(name, value)
│ │ │ │ └ [{'boxes': tensor([[ 417.6945, 667.3348, 547.8636, 767.3787],
│ │ │ │ [ 446.4780, 769.8671, 609.9299, 900.4496],
│ │ │ │ ...
│ │ │ └ 'scenario.y_pred'
│ │ └ <function Hub.update at 0x7fe202e3d0d0>
│ └ <armory.instrument.instrument.Hub object at 0x7fe22e4dd9a0>
└ <armory.instrument.instrument.Probe object at 0x7fe22e4ddf40>
File "/workspace/armory/instrument/instrument.py", line 305, in update
meter.set(arg, value, self.context["batch"])
│ │ │ │ │ └ {'batch': 0, 'stage': 'benign'}
│ │ │ │ └ <armory.instrument.instrument.Hub object at 0x7fe22e4dd9a0>
│ │ │ └ [{'boxes': tensor([[ 417.6945, 667.3348, 547.8636, 767.3787],
│ │ │ [ 446.4780, 769.8671, 609.9299, 900.4496],
│ │ │ ...
│ │ └ 'scenario.y_pred'
│ └ <function Meter.set at 0x7fe202e3d670>
└ <armory.instrument.export.ExportMeter object at 0x7fe1e9ecfeb0>
File "/workspace/armory/instrument/instrument.py", line 507, in set
self.measure()
│ └ <function ExportMeter.measure at 0x7fe202cc7ca0>
└ <armory.instrument.export.ExportMeter object at 0x7fe1e9ecfeb0>
File "/workspace/armory/instrument/export.py", line 682, in measure
self.exporter.export(
│ │ └ <function SampleExporter.export at 0x7fe202cb8e50>
│ └ <armory.instrument.export.ObjectDetectionExporter object at 0x7fe1e9ecfc10>
└ <armory.instrument.export.ExportMeter object at 0x7fe1e9ecfeb0>
File "/workspace/armory/instrument/export.py", line 31, in export
self._export(
│ └ <function ObjectDetectionExporter._export at 0x7fe202cc7310>
└ <armory.instrument.export.ObjectDetectionExporter object at 0x7fe1e9ecfc10>
File "/workspace/armory/instrument/export.py", line 125, in _export
self.image_with_boxes = self.get_sample(
│ │ └ <function ObjectDetectionExporter.get_sample at 0x7fe202cc73a0>
│ └ <armory.instrument.export.ObjectDetectionExporter object at 0x7fe1e9ecfc10>
└ <armory.instrument.export.ObjectDetectionExporter object at 0x7fe1e9ecfc10>
File "/workspace/armory/instrument/export.py", line 181, in get_sample
pred_box.astype("float32"), outline="white", width=
2
└ tensor([417.6945, 667.3348, 547.8636, 767.3787])
AttributeError: 'Tensor' object has no attribute 'astype'
2023-08-17 17:52:25 27s ERROR armory.docker.management:exec_cmd:105 command python -m armory.scenarios.main eyJfZGVzY3JpcHRpb24iOiAiQ0FSTEEgc2luZ2xlIG1vZGFsaXR5IG9iamVjdCBkZXRlY3Rpb24sIGNvbnRyaWJ1dGVkIGJ5IE1JVFJFIENvcnBvcmF0aW9uIiwgImFkaG9jIjogbnVsbCwgImF0dGFjayI6IHsia25vd2xlZGdlIjogIndoaXRlIiwgImt3YXJncyI6IHsiYmF0Y2hfc2l6ZSI6IDQsICJsZWFybmluZ19yYXRlIjogMC4wMDMsICJtYXhfaXRlciI6IDEsICJvcHRpbWl6ZXIiOiAicGdkIiwgInRhcmdldGVkIjogZmFsc2UsICJ2ZXJib3NlIjogdHJ1ZX0sICJtb2R1bGUiOiAiYXJtb3J5LmFydF9leHBlcmltZW50YWwuYXR0YWNrcy5jYXJsYV9vYmpfZGV0X2FkdmVyc2FyaWFsX3BhdGNoIiwgIm5hbWUiOiAiQ0FSTEFBZHZlcnNhcmlhbFBhdGNoUHlUb3JjaCIsICJ1c2VfbGFiZWwiOiB0cnVlfSwgImRhdGFzZXQiOiB7ImJhdGNoX3NpemUiOiAxLCAiZXZhbF9zcGxpdCI6ICJkZXYiLCAiZnJhbWV3b3JrIjogIm51bXB5IiwgIm1vZGFsaXR5IjogInJnYiIsICJtb2R1bGUiOiAiYXJtb3J5LmRhdGEuYWR2ZXJzYXJpYWxfZGF0YXNldHMiLCAibmFtZSI6ICJjYXJsYV9vdmVyX29ial9kZXRfZGV2In0sICJkZWZlbnNlIjogbnVsbCwgIm1ldHJpYyI6IHsibWVhbnMiOiB0cnVlLCAicGVydHVyYmF0aW9uIjogImwwIiwgInJlY29yZF9tZXRyaWNfcGVyX3NhbXBsZSI6IGZhbHNlLCAidGFzayI6IFsiY2FybGFfb2RfQVBfcGVyX2NsYXNzIiwgImNhcmxhX29kX2Rpc2FwcGVhcmFuY2VfcmF0ZSIsICJjYXJsYV9vZF9oYWxsdWNpbmF0aW9uc19wZXJfaW1hZ2UiLCAiY2FybGFfb2RfbWlzY2xhc3NpZmljYXRpb25fcmF0ZSIsICJjYXJsYV9vZF90cnVlX3Bvc2l0aXZlX3JhdGUiLCAib2JqZWN0X2RldGVjdGlvbl9tQVBfdGlkZSJdfSwgIm1vZGVsIjogeyJmaXQiOiBmYWxzZSwgImZpdF9rd2FyZ3MiOiB7fSwgIm1vZGVsX2t3YXJncyI6IHsibnVtX2NsYXNzZXMiOiAzLCAibW9kYWxpdHkiOiAicmdiIn0sICJtb2R1bGUiOiAicm9idXN0X29iamRldCIsICJuYW1lIjogImdldF9hcnRfbW9kZWwiLCAid2VpZ2h0c19maWxlIjogIk1JVE0tb2JqZGV0X3JnYi5wdCIsICJ3cmFwcGVyX2t3YXJncyI6IHt9fSwgInNjZW5hcmlvIjogeyJrd2FyZ3MiOiB7fSwgImV4cG9ydF9iYXRjaGVzIjogdHJ1ZSwgIm1vZHVsZSI6ICJhcm1vcnkuc2NlbmFyaW9zLmNhcmxhX29iamVjdF9kZXRlY3Rpb24iLCAibmFtZSI6ICJDYXJsYU9iamVjdERldGVjdGlvblRhc2sifSwgInN5c2NvbmZpZyI6IHsiZG9ja2VyX2ltYWdlIjogIm1pdG1fZXZhbDdfb2JqZGV0OjEuMSIsICJleHRlcm5hbF9naXRodWJfcmVwbyI6IG51bGwsICJsb2NhbF9yZXBvX3BhdGgiOiAibWl0bV9ldmFsN19vYmpkZXQiLCAiZ3B1cyI6ICJhbGwiLCAib3V0cHV0X2RpciI6IG51bGwsICJvdXRwdXRfZmlsZW5hbWUiOiBudWxsLCAidXNlX2dwdSI6IHRydWUsICJmaWxlcGF0aCI6ICIuL2V2YWw3L21pdG1fZXZhbDdfb2JqZGV0L2FkdnBhdGNoX3JnYi5qc29uIiwgIm51bV9ldmFsX2JhdGNoZXMiOiAxfSwgImV2YWxfaWQiOiAiMjAyMy0wOC0xN1QxNzUxNTguMDk1OTIwIn0= --num-eval-batches 1 --base64 did not finish cleanly
2023-08-17 17:52:25 27s INFO armory.docker.management:stop_armory_instance:141 Stopping instance: 7d8b44bd5d1b
2023-08-17 17:52:25 27s INFO armory.eval.evaluator:_cleanup:134 deleting tmp_dir /nfs/gard/aruckman/git/twosixlabs/.armory/tmp/2023-08-17T175158.095920
2023-08-17 17:52:25 27s INFO armory.eval.evaluator:_cleanup:153 results output written to:
/nfs/gard/aruckman/git/twosixlabs/.armory/outputs/2023-08-17T175158.095920/
The text was updated successfully, but these errors were encountered:
Description of the bug
Need to sanitize model output with postprocessing function to ensure it is a numpy array (assuming this is the expected type). Additionally need some more verbose type information 😅
Steps To Reproduce
Run MITM objdet with
export_batches
onAdditional Information
$ python -m armory run ./eval7/mitm_eval7_objdet/advpatch_rgb.json --num-eval-batches=1
2023-08-17 17:51:58 0s INFO armory.eval.evaluator:init:30 Constructing Evaluator Object
2023-08-17 17:51:58 0s INFO armory.eval.evaluator:init:61 Retrieving Environment Variables
2023-08-17 17:51:58 0s SUCCESS armory.docker.images:is_image_local:133 found docker image mitm_eval7_objdet:1.1
2023-08-17 17:51:59 2s INFO armory.docker.management:init:62 ARMORY Instance 7d8b44bd5d1b created.
2023-08-17 17:51:59 2s INFO armory.eval.evaluator:_run_config:287 Running evaluation script
2023-08-17 17:52:00 0s INFO armory.delayed_imports::7 Importing and configuring torch, tensorflow, and art, if available. This may take some time.
2023-08-17 17:52:04 4s INFO armory.delayed_imports::30 Setting tf.config.experimental.set_memory_growth to True on all GPUs
2023-08-17 17:52:04 4s INFO art.config:set_data_path:55 set ART_DATA_PATH to /tmp/.art/data
2023-08-17 17:52:05 5s INFO art.config:set_data_path:55 set ART_DATA_PATH to /armory/saved_models/art
2023-08-17 17:52:07 7s INFO matplotlib.font_manager:_load_fontmanager:1624 generated new fontManager
2023-08-17 17:52:11 10s INFO armory.data.utils:maybe_download_weights_from_s3:47 Using available MITM-objdet_rgb.pt in Armory
saved_model_dir
2023-08-17 17:52:16 16s INFO armory.scenarios.scenario:load_dataset:257 Loading test dataset carla_over_obj_det_dev...
2023-08-17 17:52:16 16s WARNING tensorflow.python.platform.tf_logging:warning:173 From /opt/conda/lib/python3.9/site-packages/tensorflow/python/autograph/pyct/static_analysis/liveness.py:83: Analyzer.lamba_check (from tensorflow.python.autograph.pyct.static_analysis.liveness) is deprecated and will be removed after 2023-09-23.
Instructions for updating:
Lambda fuctions will be no more assumed to be used in the statement where they are used, or at least in the same block. tensorflow/tensorflow#56089
2023-08-17 17:52:16 16s INFO armory.scenarios.scenario:evaluate_all:335 Running inference on benign and adversarial examples
Evaluation: 0% 0/1 [00:00<?, ?it/s]
Evaluation: 0% 0/1 [00:06<?, ?it/s]
2023-08-17 17:52:22 22s ERROR armory.scenarios.scenario:evaluate:452 Encountered error during scenario evaluation.
Traceback (most recent call last):
File "/opt/conda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
│ │ └ {'name': 'main', 'doc': '\nMain script for running scenarios. Users will run a scenario by calling:\n armory r...
│ └ <code object at 0x7fe22f6bb240, file "/workspace/armory/scenarios/main.py", line 1>
└ <function _run_code at 0x7fe31a85f9d0>
File "/opt/conda/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
│ └ {'name': 'main', 'doc': '\nMain script for running scenarios. Users will run a scenario by calling:\n armory r...
└ <code object at 0x7fe22f6bb240, file "/workspace/armory/scenarios/main.py", line 1>
File "/workspace/armory/scenarios/main.py", line 229, in
run_config(
└ <function run_config at 0x7fe22e52e1f0>
File "/workspace/armory/scenarios/main.py", line 145, in run_config
scenario.evaluate()
│ └ <function Scenario.evaluate at 0x7fe202cdf670>
└ <armory.scenarios.carla_object_detection.CarlaObjectDetectionTask object at 0x7fe22f385730>
The text was updated successfully, but these errors were encountered: