[TOPI][IMAGE][RESIZE] Bilinear interpolation for resize and upsampling.

[srkreddy1238]

    * upsampling - migrated to cpp
    * bilinear resize implementation & test cases.
    * upsampling testcases enhanced for NHWC

Discuss:
Above implementation merges both nn (upsampling) and bilinear at topi.cpp level as scale.
How about merging upsampling & bilinear as scale at python tvm.topi and nnvm front end too ?

We can have just "scale" at all places in this case.

[FrozenGene]

I just go through in mobile. Maybe omit something.
Where is align_corners attribue? for tf, align_corners will affect the h_scale / w_scale.

[srkreddy1238]

align_corners doesn't affect the final shape, it just few pixels retained from input source to output with out re computation..
I am just trying to understand align_corners logic for both nn and bilinear.

I will add that soon while review in progress.

[srkreddy1238]

How about merging upsampling and bilinear into scale from frontend ?

[FrozenGene]

align_corners, you can refer the TF's implementation:

For nearest neighbours:

const int64 in_y = std::min(
            (align_corners) ? static_cast<int64>(roundf(y * height_scale))
                            : static_cast<int64>(floorf(y * height_scale)),
            in_height - 1);

for (int x = 0; x < out_width; ++x) {
          const int64 in_x = std::min(
              (align_corners) ? static_cast<int64>(roundf(x * width_scale))
                              : static_cast<int64>(floorf(x * width_scale)),
              in_width - 1);

For ResizeBilinear:

// CalculateResizeScale determines the float scaling factor.
inline float CalculateResizeScale(int64 in_size, int64 out_size,
                                  bool align_corners) {
  return (align_corners && out_size > 1)
             ? (in_size - 1) / static_cast<float>(out_size - 1)
             : in_size / static_cast<float>(out_size);
}
height_scale = CalculateResizeScale(in_height, out_height, align_corners_);
width_scale = CalculateResizeScale(in_width, out_width, align_corners_);

I personally prefer keep separate interface. Which will make the customer know what op do. How to implement these op is the low-level framework's work. You can unify into scale you can also choose not to do.

[srkreddy1238]

Thanks, I will check the above logic for align_coeners.

Separation at customer end happens by "mode" attribute. The template and purpose is same hence I feel good to unify upto frontend.

@tqchen what do you think ?

[srkreddy1238]

@FrozenGene

I got through align_corners for bilinear, the same for nn is a challenge due to round, floor compatible IR instructions.
I tried with 'cast' and addition of 0.5f which worked for llvm but failed on cuda.

Any ideas ? Or will hold align_corners for nn until there is a demand for it !

[srkreddy1238]

• @masahi for review.

[FrozenGene]

You can implement floor like this: https://discuss.tvm.ai/t/floordiv-operation-in-tensors-how-to-achieve/146/5 Then we should not have not-compatibility instruction

[srkreddy1238]

Ok, let me try the floor operator addition first. I hope mod division work fine with all platforms by default.
round would be easy if we have floor [ round(f) = floor(f+0.5) ].

[masahi]

thanks, having bilinear upsampling is nice. But what is the use case of asymmetric upsampling (different scales for width and height) ?

[srkreddy1238]

Yes, different scale for width and height.

I don't have information on real time models for asymmetric scale.
I am trying to unify the API for all algorithms instead of having multiple operators from frontend to topi.
Unified template would be
scale(data, out_shape, layout='NCHW', align_corners=False, mode='NN')

I felt its easy to maintain with new algorithms in future if required.

Thanks for the review :)

[FrozenGene]

Please do the test on the OpenCL platform carefully. Someone told me that mod opeartor on floating point type has some issue.

[srkreddy1238]

Sure, I will check that too.

[tqchen]

any updates on this PR? @masahi @FrozenGene do you have followup comments?

[masahi]

Shouldn't upsampling and bilinear_scale be merged into a single API? I think if we have bilinear scaling available, then upsampling op should support bilinear upsampling as well.

[srkreddy1238]

Can I merge them both as "scale" and made it generic at all levels?

[srkreddy1238]

@masahi
I mean API "scale" supporting both up/down sampling.

[masahi]

I think "upsampling" is a better name than "scale". I don't know why you want down sampling, when we already have pooling.

[srkreddy1238]

@tqchen & @masahi

• Weights compute made inline inside topi.
• Extra parameter handling removed across.

@tqchen
BTW, is CI still on python2 ?

[tqchen]

CI is upgraded to use python3 in most cases but there is still python2 checks, so the code is better to be compatible until 2019

[masahi]

not needed anymore

[srkreddy1238]

Thanks. removed now.

[masahi]

LGTM.

[tqchen]

Thanks to the reviewers and the contributor, this is merged.

[ppwwyyxx]

From the discussion it seems like this implementation follows resize in tensorflow.

However, resize(align_corners=False) in tensorflow has bugs since day 1 and it was fixed in Feb this year by introducing a new resize operator in TF. See tensorflow/tensorflow#6720.
The fixed version of align_corners=False agree with pytorch & opencv, etc. It would be good to fix it in topi.