演示DEMO

​1. DEMO概述

当前SDK内有关于模型转换的一些演示demo:

示例目录	功能	网络模型	输入	输出
create_lmdb_demo	基于imageset创建lmdb文件	无	image set	lmdb
classify_demo	分类网络模型转换以及推理流程	resnet18	fp32umodel	int8 umodel
detect_demo	人脸检测网络模型转换以及推理流程	squeezenet_21k	fp32umodel	int8 umodel
googlenet_bmodel_convert	googlenet转换为int8 bmodel完整流程	googlenet	fp32 caffemodel	int8 bmodel
resnet50_bmodel_convert	resnet50　caffemodel转换为int8 bmodel完成流程	resnet50	fp32 caffemodel	int8 bmodel

2. create_lmdb_demo

create_lmdb_demo文件路径为： /workspace/examples/calibration/create_lmdb_demo demo主要描述了如何使用convert_imageset来生成量化所使用的lmdb数据集合：

  function convert_imageset_to_lmdb_demo()
  {
    rm $IMG_DIR/img_lmdb -rif

    cali_gen_imagelist; ret=$?;
    if [ $ret -ne 0 ]; then echo "#ERROR: cali_gen_imagelist"; return $ret; fi

    # argv[1] : image dir
    # argv[2] : image list
    convert_imageset --shuffle --resize_height=256 --resize_width=256 \
      $IMG_DIR/  $IMG_DIR/ImgList.txt  $IMG_DIR/img_lmdb
    ret=$?; if [ $ret -ne 0 ]; then echo "#ERROR: convt imgage to lmdb"; fi
    echo "#INFO: Convert Images to lmdb done"
    return $ret;
  }

其中resize_width需要根据网络模型的实际输入大小来确定 在此路径下，直接执行启动脚本，可以生成对应的lmdb:

  root@cai-thinkpad-t470p:/workspace/examples/calibration/create_lmdb_demo# ./create_lmdb_demo.sh 
  /workspace/examples/calibration/create_lmdb_demo/images /workspace/examples/calibration/create_lmdb_demo
  #INFO: Create Imageset Done
  /workspace/examples/calibration/create_lmdb_demo
  I1008 14:43:46.345486   232 convert_imageset.cpp:86] Shuffling data
  I1008 14:43:46.345930   232 convert_imageset.cpp:89] A total of 2 images.
  I1008 14:43:46.346113   232 db_lmdb.cpp:35] Opened lmdb ./images//img_lmdb
  I1008 14:43:46.370638   232 convert_imageset.cpp:153] Processed 2 files.
  #INFO: Convert Images to lmdb done
  root@cai-thinkpad-t470p:/workspace/examples/calibration/create_lmdb_demo# tree
  .
  |-- create_lmdb_demo.sh
  `-- images
      |-- ImgList.txt
      |-- cat\ gray.jpg
      |-- cat.jpg
      |-- cat_gray.jpg
      |-- fish-bike.jpg
      `-- img_lmdb
          |-- data.mdb
          `-- lock.mdb

其中的images/img_lmdb就是生成的lmdb数据集。用户可以参考此示例生成模型对应的数据集合。

3. classify_demo

主要是使用resnet18.caffemodel如何生成对应的int8 umodel，并验证int8下的精度。 其中自带lmdb文件，并且校准迭代次数简化为10次，方便快速执行：

   root@cai-thinkpad-t470p:/workspace/examples/calibration/classify_demo# ./classify_demo.sh 
   FLAGS_iterations = 10
   FLAGS_BITWITH = TO_INT8
   I1008 14:50:15.044811   242 common.cpp:345] use_GPU have not be set !
   I1008 14:50:15.045114   242 calibration_use_pb.cpp:499] Use CPU.
   I1008 14:50:15.056399   242 net.cpp:1420] Initializing net from parameters: 
   name: "ResNet-18"
   state {
     phase: TEST
   }
   layer {
     name: "data"
     type: "Data"
     top: "data"
     top: "label"
     include {
       phase: TEST
     }
     transform_param {
       mirror: false
       crop_size: 224
       mean_value: 103.939
       mean_value: 116.779
       mean_value: 123.68
     }
     data_param {
       source: "./lmdb/imagenet_s/ilsvrc12_val_lmdb"
       batch_size: 1
       backend: LMDB
     }
   }
   ...
   I1008 14:50:34.734228   245 accuracy_layer.cpp:101] use un-unique top_k
   I1008 14:50:34.734349   245 ufw.cpp:297] Batch 9, acc/top-1 = 0
   I1008 14:50:34.734374   245 ufw.cpp:297] Batch 9, acc/top-5 = 1
   I1008 14:50:34.734385   245 ufw.cpp:302] Batch 9 Finished 
   I1008 14:50:34.734391   245 ufw.cpp:305] Loss: 0
   I1008 14:50:34.734407   245 ufw.cpp:313] acc/top-5 = 0.9
   I1008 14:50:34.734426   245 ufw.cpp:313] acc/top-1 = 0.6
   #INFO: Test Resnet-INT8 Done

执行后打印上述top_k的置信度。同时生成了对应的umodel:

   root@cai-thinkpad-t470p:/workspace/examples/calibration/classify_demo# tree
   .
   |-- classify_demo.sh
   |-- lmdb
   |   `-- imagenet_s
   |       `-- ilsvrc12_val_lmdb
   |           |-- data.mdb
   |           `-- lock.mdb
   |-- models
   |   |-- resnet18.fp32umodel
   |   |-- resnet18.int8umodel
   |   |-- resnet18.prototxt
   |   |-- resnet18_deploy_fp32_unique_top.prototxt
   |   |-- resnet18_deploy_int8_unique_top.prototxt
   |   |-- resnet18_test_fp32_unique_top.prototxt
   |   `-- resnet18_test_int8_unique_top.prototxt
   `-- resnet18_test_int8_unique_top_accuracy_int8.md

4. detect_demo

主要使用squeezenet网络来做detect演示，主要包括：

1. 检测程序编译。

2. squeezennet int8模型的量化生成。

3. 利用int8模型进行人脸检测。 用户可以根据需要参考face_demo.sh文件的最后几行执行命令。

   root@cai-thinkpad-t470p:/workspace/examples/calibration/detect_demo# ./face_demo.sh 
   FLAGS_iterations = 100
   FLAGS_BITWITH = TO_INT8
   I1008 14:59:36.823330   257 common.cpp:345] use_GPU have not be set !
   I1008 14:59:36.823693   257 calibration_use_pb.cpp:499] Use CPU.
   I1008 14:59:36.831105   257 upgrade_proto.cpp:69] Attempting to upgrade input file specified using deprecated input fields: models/squeezenet/squeezenet_21k_test.prototxt
   I1008 14:59:36.831171   257 upgrade_proto.cpp:72] Successfully upgraded file specified using deprecated input fields.
   W1008 14:59:36.831178   257 upgrade_proto.cpp:74] Note that future Ufw releases will only support input layers and not input fields.
   I1008 14:59:36.839711   257 net.cpp:1420] Initializing net from parameters: 
   name: "SqueezeNet"
   state {
     phase: TEST
   }
   layer {
     name: "data"
     type: "Data"
     top: "data"
     top: "label"
     include {
       phase: TEST
     }
     transform_param {
       mean_value: 104
       mean_value: 117
       mean_value: 123
     }
     data_param {
       source: "./lmdb/100_pics_squeezenet_711_400_lmdb"
       batch_size: 1
       backend: LMDB
     }
   }
   ...
   I1008 15:07:12.407119   284 layer.hpp:481] UFW INT8_NEURON MODEL
   final predict 19 bboxes
   ...

大约10分钟后，执行完毕，使用in8umodel最终检测出19个目标框，可以查看生成的图片确认。另外可以修改face_demo.sh中的执行命令，测试fp32网络的检测结果，二者可以比对。

5. googlenet_bmodel_convert

此示例包含下载作者版模型，并且脚本一键转换模型，用户可以直接基于此demo框架添加私有模型，并修改脚本内对应参数，既可以从caffemodel转成部署使用的bmodel:

root@cai-thinkpad-t470p:/workspace/examples/calibration/googlenet_bmodel_convert# ./download_resnet50_model.sh
root@cai-thinkpad-t470p:/workspace/examples/calibration/googlenet_bmodel_convert# ./generate_int8_bmodel.sh

function convert_to_int8_umodel()
{
  calibration_use_pb release \
    -model=./models/googlenet-quantize.prototxt   \
    -weights=./models/bvlc-googlenet.caffemodel \
    -iterations=${CALI_NUM} \
    -bitwidth=TO_INT8

  ret=$?; if [ $ret -ne 0 ]; then echo "#ERROR: googlenet Fp32ToInt8"; popd; return $ret;fi

  echo "#INFO: Run Example (googlenet Fp32ToInt8) Done"
  return $ret;
}

function convert_to_bmodel()
{
  mkdir -p int8model
  bmnetu -model ./models/bvlc-googlenet_deploy_int8_unique_top.prototxt \
       -weight ./models/bvlc-googlenet.int8umodel \
       -max_n 1 \
       -prec=INT8 \
       -dyn=0 \
       -cmp=1 \
       -target=BM1684 \
       -outdir=./int8model
}

最终会生成所需bmodel:

|-- int8model
|   |-- compilation.bmodel
|   |-- compiler_profile_0.dat
|   |-- input_ref_data.dat
|   `-- output_ref_data.dat

并且会验证生成的bmodel的精度是否与原caffemodel一致(bmrt_test)：

******do bmodel test******
[BMRT][deal_with_options:901] INFO : Loop num: 1
begin to cmodel init...
mult engine c_model init
l2_fill complete
npu num = 64
eu num = 32
global mem size = 4294967296
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init 
[BMRT][load_bmodel:660] INFO : Loading bmodel from [./int8model/compilation.bmodel]. Thanks for your patience...
[BMRT][load_bmodel:642] INFO : pre net num: 0, load net num: 1
[BMRT][bmrt_test:503] INFO : ==> running network #0, name: ResNet-50, loop: 0
[BMRT][bmrt_test:614] INFO : net[ResNet-50] stage[0], launch total time is 5297646 us (npu 0 us, cpu 5297646 us)
[BMRT][bmrt_test:650] INFO : +++ The network[ResNet-50] stage[0] cmp success +++
[BMRT][bmrt_test:674] INFO : load input time(s): 0.000130
[BMRT][bmrt_test:675] INFO : calculate  time(s): 5.297647
[BMRT][bmrt_test:676] INFO : get output time(s): 0.000015
[BMRT][bmrt_test:677] INFO : compare    time(s): 0.000044
BMLIB Send Quit Message

为方便演示，自带lmdb文件，为Imagenet.

6. resnet50_bmodel_convert

此demo与googlenet的量化类似，用户可以直接基于此demo框架添加私有模型，并修改脚本内对应参数，既可以从caffemodel转成部署使用的bmodel:

function convert_to_int8_umodel()
{
  calibration_use_pb release \
    -model=./models/ResNet-50-quantize.prototxt   \
    -weights=./models/ResNet-50-model.caffemodel \
    -iterations=${CALI_NUM} \
    -bitwidth=TO_INT8

  ret=$?; if [ $ret -ne 0 ]; then echo "#ERROR: Resnet50 Fp32ToInt8"; popd; return $ret;fi

  echo "#INFO: Run Example (Resnet18 Fp32ToInt8) Done"
  return $ret;
}

function convert_to_bmodel()
{
  mkdir -p int8model
  bmnetu -model ./models/ResNet-50-model_test_int8_unique_top.prototxt \
       -weight ./models/ResNet-50-model.int8umodel \
       -max_n 1 \
       -prec=INT8 \
       -dyn=0 \
       -cmp=1 \
       -target=BM1684 \
       -outdir=./int8model
}

并且会验证生成的bmodel的精度是否与原caffemodel一致(bmrt_test)：

******do bmodel test******
[BMRT][deal_with_options:901] INFO : Loop num: 1
begin to cmodel init...
mult engine c_model init
l2_fill complete
npu num = 64
eu num = 32
global mem size = 4294967296
bmcpu init: skip cpu_user_defined
open usercpu.so, init user_cpu_init 
[BMRT][load_bmodel:660] INFO : Loading bmodel from [./int8model/compilation.bmodel]. Thanks for your patience...
[BMRT][load_bmodel:642] INFO : pre net num: 0, load net num: 1
[BMRT][bmrt_test:503] INFO : ==> running network #0, name: googlenet, loop: 0
[BMRT][bmrt_test:614] INFO : net[googlenet] stage[0], launch total time is 8462144 us (npu 0 us, cpu 8462144 us)
[BMRT][bmrt_test:650] INFO : +++ The network[googlenet] stage[0] cmp success +++
[BMRT][bmrt_test:674] INFO : load input time(s): 0.000133
[BMRT][bmrt_test:675] INFO : calculate  time(s): 8.462144
[BMRT][bmrt_test:676] INFO : get output time(s): 0.000016
[BMRT][bmrt_test:677] INFO : compare    time(s): 0.000045
BMLIB Send Quit Message

为方便演示，自带lmdb文件，为Imagenet.