[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for TensorRT 10 #68715

Draft
wants to merge 12 commits into
base: master
Choose a base branch
from
+1,096 −131
Draft

Add support for TensorRT 10 #68715

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
94d1bb9
[TRT10] Add support for TensorRT 10.0
meena-at-work Mar 27, 2024
864671b
Use same inc file for TRT10 as for TRT8 and below
meena-at-work Apr 4, 2024
fc5523f
Update TF-TRT to support TensorRT 10.0 APIs
benbarsdell Apr 23, 2024
711dfa5
Fix TF-TRT shape layer int64/int32 mismatches
benbarsdell May 16, 2024
b81ae11
Disable implicit batch in TF-TRT tests for TRT10
benbarsdell May 16, 2024
be1316f
Disable explicit precision in TF-TRT with TRT10
benbarsdell May 16, 2024
e99530e
Fix bug in TF-TRT with TRT10 when finding engines
benbarsdell May 16, 2024
5131ab9
Disable implicit batch tests in TrtConverterTest
benbarsdell May 16, 2024
a0f1c62
Disable implicit batch tests in TrtEngineOpTest
benbarsdell May 21, 2024
ed93558
Disable implicit batch in TRT convert nodes test
benbarsdell May 21, 2024
210c7b1
Fix swapped "are"/"are not" in TRT test utils
benbarsdell May 28, 2024
699a77b
Change TRT to default to use_dynamic_shape=True
benbarsdell May 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Change TRT to default to use_dynamic_shape=True 
- use_dynamic_shape=False is not supported since TensorRT 10.0.
- Also expands the related error message.
  • Loading branch information
@benbarsdell
benbarsdell committed May 28, 2024
commit 699a77b12fc9b061b05534a87468bf44c7206e55
3 changes: 2 additions & 1 deletion tensorflow/compiler/tf2tensorrt/kernels/trt_engine_op.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1288,7 +1288,8 @@ StatusOr<std::pair<EngineContext*, int>> TRTEngineOp::GetEngine(
}
#else
return errors::Internal(
"Implicit batch is not supported since TensorRT 10.0");
"Implicit batch is not supported since TensorRT 10.0. Pass "
"use_dynamic_shape=True to TrtGraphConverterV2 to avoid this error.");
#endif
}

Expand Down
4 changes: 2 additions & 2 deletions tensorflow/python/compiler/tensorrt/trt_convert.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1214,7 +1214,7 @@ def __init__(self,
input_saved_model_signature_key: the key of the signature to optimize the
graph for.
use_dynamic_shape: whether to enable dynamic shape support. None is
equivalent to False in the current implementation.
equivalent to True in the current implementation.
dynamic_shape_profile_strategy: one of the strings in
supported_profile_strategies(). None is equivalent to Range in the
current implementation.
Expand Down Expand Up @@ -1284,7 +1284,7 @@ def __init__(self,
self._calibrated = False

if use_dynamic_shape is None:
self._use_dynamic_shape = False
self._use_dynamic_shape = True
else:
self._use_dynamic_shape = use_dynamic_shape

Expand Down
Loading