:j3rddlZddlZddlZddlZddlmZ ddZdZdZdZ dZ ddZ d Z d Z d Zed k(r{eZed eddlmZeej(ej*sej,sJdeeZn ej*ZddlmZeeeZeD] Zee yy)N) TensorProtoc^tj}|jdddtd|jdddtd|jd d dtd d |jdddtdd |jddtd d |jddtd d |jddtdd |jddt dd |jddtdd |jddtdd |jd!dtdd" |jd#dtdd$ |jd%dd&gd'd()|jd*d+dd,d-.|j d/|jd0dtd1d2 |jd3dd,d4.|j d5|jd6dd,d7.|j d8|jd9d:dd,;|j d<|j|S)=Nz-iz--inputFz2Set the input file for reading the profile results)requiredtypehelpz-mz--modelzIonnx model path to run profiling. Required when --input is not specified.z-bz --batch_sizezbatch size of input)rrdefaultrz-sz--sequence_length zsequence length of inputz--past_sequence_lengthzpast sequence length for gpt2z--global_lengthz&number of global tokens for longformerz --samplesiz\number of samples to test. Set it large enough to reduce the variance of performance result.z --thresholdg{Gz?zfThreshold of run time ratio among all nodes. Nodes with larger ratio will show in top expensive nodes.z --thread_numznumber of threads to usez--input_ids_namez"input name for input IDs, for bertz--segment_ids_namez$input name for segment IDs, for bertz--input_mask_namez'input name for attention mask, for bertz--dummy_inputsr )bertgpt2 longformerr zEType of model inputs. The default will create dummy inputs with ones.)rr choicesrz-gz --use_gpu store_truezuse GPU)ractionr)use_gpuz --providercudazExecution provider to usez--basic_optimizationz_Enable only basic graph optimizations. By default, all optimizations are enabled in OnnxRuntime)basic_optimizationz--kernel_time_onlyz.Only include the kernel time and no fence time)kernel_time_onlyz-vz --verbose)rr)verbose)argparseArgumentParser add_argumentstrintfloat set_defaults parse_args)argvparsers v/home/mcse/projects/srt_converter/srt-converter-venv/lib/python3.12/site-packages/onnxruntime/transformers/profiler.pyparse_argumentsr"s  $ $ &F    A     X     "     '     ,    5    k    u    '    1    3    6  9 T  kE,U^_ &   (   n   51  =   / kE,O &   T ""cddlm}||||| |d}|D]}|jd|} |j} | S)Nr)create_onnxruntime_sessionT)enable_all_optimization num_threadsenable_profiling)benchmark_helperr%run end_profiling) onnx_model_pathrproviderr thread_num all_inputsr%sessioninputs_ profile_files r! run_profiler4s[;($6 6 G& KKf %&((*L r#c~t|jdtk(rt||jdSdS)Nvalue)r WhichOneofrgetattr)dims r!get_dim_from_type_protor:s4489P4QUX4X73w/ 0b^bbr#cz|jjjDcgc] }t|c}Scc}wN) tensor_typeshaper9r:) type_protods r!get_shape_from_type_protorAs.0:0F0F0L0L0P0P Q1 #A & QQ Qs8c i}|jD]Z}t|j}g}t|D]'\}} t | t s|j |)t|dkDryt|dkDr|||d<t|dkDr|||d<|jjj} | tjtjtjfvsJ| tjk(rtjn2| tjk(rtj ntj"} tj$|| } | ||j&<]t)|D cgc]} |}} |Scc} w)a Create dummy inputs for ONNX model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples Returns: List[Dict]: list of inputs Nrrdtype)'get_graph_inputs_excluding_initializersrAr enumerate isinstancerappendlenr= elem_typerFLOATINT32INT64numpyfloat32int64int32onesnamerange) onnx_model batch_sizesequence_lengthsamples dummy_inputs graph_inputr> symbol_dimsir9rK data_typedatar2r/s r!create_dummy_inputsr`sjL!IIK. )+*:*:; & &FAs#s#""1% & { a  { a $.E+a. ! { a $3E+a. !$$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  zz%y1)- [%%&/.2).g71,7J7 8s> F c Vddlm}m}|||||\} } } ||||dd| | | d } | S)a-Create dummy inputs for BERT model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: List[Dict]: list of inputs r)find_bert_inputsgenerate_test_data{F) test_casesseedr input_ids segment_ids input_maskrandom_mask_length)bert_test_datarbrc) rVrWrXrYinput_ids_namesegment_ids_nameinput_mask_namerbrcrgrhrir/s r!create_bert_inputsrosN.D)9*nVfhw)x&I{J#   J r#c|||||zd}i}|jD]%}t|j}t|D]0\} } t | t s| |vrt d| || || <2|jjj} | tjtjtjfvsJ| tjk(rtjn2| tjk(rtjntj } tj"|| } | ||j$<(t'|Dcgc]}|}}|Scc}w)aCreate dummy inputs for GPT-2 model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length past_sequence_length (int): past sequence length samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs )rWseq_len past_seq_len total_seq_lensymbol is not supported: rD)rFrArrGrHr RuntimeErrorr=rKrrLrMrNrOrPrQrRrSrTrU)rVrWrXpast_sequence_lengthrYsymbolsrZr[r>r]r9rKr^r_r2r/s r!create_gpt2_inputsrx sT$!",(+?? GL!IIK. )+*:*:;& ,FAs#s#g%&)B3%'HII&s|E!H  , $$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  zz%y1)- [%%&#.&).g71,7J7 8s E c"||d}i}|jD]V}t|j}t|D]0\} } t | t s| |vrt d| || || <2|jjj} | tjtjtjfvsJ| tjk(rtjn2| tjk(rtjntj } d|j"vr#tj$|| } d| ddd|f<ntj&|| } | ||j"<Yt)|Dcgc]}|}}|Scc}w)aCreate dummy inputs for Longformer model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length global_length (int): number of global tokens samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_longformer_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs )rWrXrtglobalrDrN)rFrArrGrHrrur=rKrrLrMrNrOrPrQrRrTzerosrSrU)rVrWrX global_lengthrYrwrZr[r>r]r9rKr^r_r2r/s r!create_longformer_inputsr};st (OLGL!IIK. )+*:*:;& ,FAs#s#g%&)B3%'HII&s|E!H  , $$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  {'' ';;uI6D&'DN]N" #::e95D)- [%%&-.0).g71,7J7 8s? F c |jdkDr |jntjd}dtjvrt |tjd<ddlm}ddlm }|||j}d}|jdk(rNt||j|j|j|j |j"|j$}n|jdk(r8t'||j|j|j(|j}ns|jd k(r8t+||j|j|j,|j}n,t/||j|j|j}t1|j|j2|j4|j6|j|}|S) NrF)logicalOMP_NUM_THREADS)load) OnnxModelr r r)r.psutil cpu_countosenvironronnxrrVrmodelrZrorWrXrYrlrmrnrxrvr}r|r`r4rr-r)argsr'rrrVr/r3s r!r*r*js%)__q%8$//f>N>NW\>]K *(+K(8 $%$4 +,JJ F"'  OO  LL     ! !      f $'  OO   % % LL     l *-  OO     LL  )T__dFZFZ\`\h\hi       L r#__main__ Arguments) setup_loggerzMrequires either --model to run profiling or --input to read profiling results)process_resultsr<)NNN)rrrOrrrr"r4r:rAr`rorxr}r*__name__ argumentsprintr)rrinputrr3profile_result_processorrresultsliner#r!rs I#X&cR'^&R-`,^3l z!I +y!-""# ??o oo9~   8lI6G d #r#