[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question]: fp16推理的效率问题 #5285

Open
lddcdut opened this issue Mar 17, 2023 · 0 comments
Open

[Question]: fp16推理的效率问题 #5285

lddcdut opened this issue Mar 17, 2023 · 0 comments
Assignees
Labels
question Further information is requested triage

Comments

@lddcdut
Copy link
lddcdut commented Mar 17, 2023

请提出你的问题

环境:

onnxconverter-common      1.13.0
onnxruntime-gpu           1.13.1
paddlenlp                 2.5.2.post0
paddlepaddle-gpu          2.4.2.post116
显卡:RTX3060 12G

写了一段测试代码,用于比较fp16和fp32的效率

schema = ["标签1", "标签2", "标签3", "标签4", "标签5", "标签6", "标签7", "标签8", "标签9", "标签10", "标签11", "标签12",
          "标签13", "标签14", "标签15", "标签16", "标签17", "标签18", "标签19", "标签20"]
ie = Taskflow('information_extraction', schema=schema)
for x in range(10):
    start_time = time.time()
    ie("威尔哥(Virgo)减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制,1956年开始研制,1963年进入服役,装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机,主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。威尔哥(Virgo)减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制,1956年开始研制,1963年进入服役,装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机,主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。")
    end_time = time.time()
    print(f'用时:{(end_time - start_time) * 1000}')

ie = Taskflow('information_extraction', schema=schema, precision='fp16')
for x in range(10):
    start_time = time.time()
    ie("威尔哥(Virgo)减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制,1956年开始研制,1963年进入服役,装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机,主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。威尔哥(Virgo)减速炸弹是由瑞典FFV军械公司专门为瑞典皇家空军的攻击机实施低空高速轰炸而研制,1956年开始研制,1963年进入服役,装备于A32“矛盾”、A35“龙”、和AJ134“雷”攻击机,主要用于攻击登陆艇、停放的飞机、高炮、野战火炮、轻型防护装甲车辆以及有生力量。")
    end_time = time.time()
    print(f'用时:{(end_time - start_time) * 1000}')

测试结果发现差别不大,这种现象正常吗,看文档速度应该翻倍才对啊

fp32

用时:1368.5643672943115
用时:393.4502601623535
用时:395.1888084411621
用时:390.81835746765137
用时:387.7217769622803
用时:393.5997486114502
用时:391.2484645843506
用时:390.1629447937012
用时:395.39194107055664
用时:393.71204376220703

fp16

用时:593.8384532928467
用时:361.1023426055908
用时:339.4172191619873
用时:367.8784370422363
用时:358.5941791534424
用时:363.76333236694336
用时:377.02012062072754
用时:361.52172088623047
用时:367.0346736907959

image

@lddcdut lddcdut added the question Further information is requested label Mar 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested triage
Projects
None yet
Development

No branches or pull requests

2 participants