Step-by-Step
============

This document list steps of reproducing Intel Optimized TensorFlow image recognition models tuning results via LPOT.

> **Note**: 
> Most of those models are both supported in Intel optimized TF 1.15.x and Intel optimized TF 2.x.
> [Version support](../../../README.md)
# Prerequisite

### 1. Installation
  Recommend python 3.6 or higher version.

  ```shell
  cd examples/tensorflow/image_recognition
  pip install -r requirements.txt
  ```

### 2. Prepare Dataset
Please use the script under the folder `datasets` to download and convert PASCAL VOC 2012 semantic segmentation dataset to TFRecord. Refer to [Running DeepLab on PASCAL VOC 2012 Semantic Segmentation Dataset](https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/pascal.md#running-deeplab-on-pascal-voc-2012-semantic-segmentation-dataset) for more details.
```shell
# From the examples/tensorflow/semantic_image_segmentation/deeplab/datasets directory.
sh download_and_convert_voc2012.sh
```

### 3. Prepare pre-trained model
Refer to [Export trained deeplab model to frozen inference graph](https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/export_model.md#export-trained-deeplab-model-to-frozen-inference-graph) for more details.

1. Download the checkpoint file
```shell
wget http://download.tensorflow.org/models/deeplabv3_pascal_train_aug_2018_01_04.tar.gz
tar -xvf deeplabv3_pascal_train_aug_2018_01_04.tar.gz
```
2. Export to a frozen graph
```shell
git clone https://github.com/tensorflow/models.git
cd models/research/
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

python deeplab/export_model.py \ 
    --checkpoint_path=/PATH/TO/deeplabv3_pascal_train_aug/model.ckpt \ 
    --export_path=/PATH/TO/deeplab_export.pb \ 
    --model_variant="xception_65"  \ 
    --logtostderr \ 
    --eval_split="val" \ 
    --model_variant="xception_65" \ 
    --atrous_rates=6 \ 
    --atrous_rates=12 \ 
    --atrous_rates=18 \ 
    --output_stride=16 \ 
    --decoder_output_stride=4 \ 
    --eval_crop_size="513,513" \ 
    --dataset="pascal_voc_seg"
```

# Run
```shell
cd examples/tensorflow/semantic_image_segmentation/deeplab
bash run_tuning.sh --config=deeplab.yaml --input_model=/PATH/TO/deeplab_export.pb --output_model=./lpot_deeplab.pb
```


Examples of enabling Intel® Low Precision Optimization Tool auto tuning on Deeplab model for tensorflow
=======================================================

This is a tutorial of how to enable deeplab model with Intel® Low Precision Optimization Tool.

# User Code Analysis

Intel® Low Precision Optimization Tool supports two usages:

1. User specifies fp32 "model", yaml configured calibration dataloader in calibration field and evaluation dataloader in evaluation field, metric in tuning.metric field of model-specific yaml config file.

> *Note*: 
> you should change the model-specific yaml file dataset path to your own dataset path

2. User specifies fp32 "model", calibration dataset "q_dataloader" and a custom "eval_func" which encapsulates the evaluation dataset and metric by itself.

We provide Deeplab model pretrained on PASCAL VOC 2012, Using mIOU as metric which is built-in supported by Intel® Low Precision Optimization Tool.

### Write Yaml config file

In examples directory, there is a template.yaml. We could remove most of the items and only keep mandatory item for tuning. 


```yaml
# deeplab.yaml

model:                                               # mandatory. lpot uses this model name and framework name to decide where to save tuning history and deploy yaml.
  name: deeplab
  framework: tensorflow                              # mandatory. supported values are tensorflow, pytorch, pytorch_ipex, onnxrt_integer, onnxrt_qlinear or mxnet; allow new framework backend extension.

quantization:                                        # optional. tuning constraints on model-wise for advance user to reduce tuning space.
  calibration:
    sampling_size: 50, 100                           # optional. default value is 100. used to set how many samples should be used in calibration.
    dataloader:
      batch_size: 1
      dataset:
        VOCRecord:
          root: /path/to/pascal_voc_seg/tfrecord     # NOTE: modify to calibration 
      transform:
        ParseDecodeVoc: {}


evaluation:                                          # optional. required if user doesn't provide eval_func in lpot.Quantization.
  accuracy:                                          # optional. required if user doesn't provide eval_func in lpot.Quantization.
    metric:
      mIOU: 
        num_classes: 21                              # built-in metrics are topk, map, f1, allow user to register new metric.
    dataloader:
      batch_size: 1
      dataset:
        VOCRecord:
          root: /path/to/pascal_voc_seg/tfrecord     # NOTE: modify to evaluation dataset location if needed
      transform:
        ParseDecodeVoc: {}
  performance:                                       # optional. used to benchmark performance of passing model.
    iteration: 100
    configs:
      cores_per_instance: 4
      num_of_instance: 6
    dataloader:
      batch_size: 1
      dataset:
        VOCRecord:
          root: /path/to/pascal_voc_seg/tfrecord     # NOTE: modify to evaluation dataset location if needed
      transform:
        ParseDecodeVoc: {}

tuning:
  accuracy_criterion:
    relative:  0.01                                  # optional. default value is relative, other value is absolute. this example allows relative accuracy loss: 1%.
  exit_policy:
    timeout: 0                                       # optional. tuning timeout (seconds). default value is 0 which means early stop. combine with max_trials field to decide when to exit.
  random_seed: 9527                                  # optional. random seed for deterministic tuning.


```

Here we choose topk which is built-in metric and set accuracy criterion as tolerating 0.01 relative accuracy loss of baseline. The default tuning strategy is basic strategy. The timeout 0 means early stop as long as a tuning config meet accuracy target.

### Tune

After completed preparation steps, we just need to add below tuning part in `eval_classifier_optimized_graph` class.

```python
from lpot.experimental import Quantization, common
quantizer = Quantization(self.args.config)
quantizer.model = common.Model(self.args.input_graph)
q_model = quantizer()
q_model.save(self.args.output_graph)
```

### Benchmark
```python
from lpot.experimental import Benchmark, common
evaluator = Benchmark(self.args.config)
evaluator.model = common.Model(self.args.input_graph)
evaluator(self.args.mode)
```

