Skip to content

[HOLD] [$250] [MEDIUM] Smartscan: Receipt thumbnail should show the top of the receipt #34120

Closed

Description

@m-natarajan

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!


Version Number: 1.4.22-6
Reproducible in staging?: yes
Reproducible in production?:.yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @quinthar
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1704757554079199

Action Performed:

  1. create an IOU using receipt scanning
  2. Observe the thumbnail of the receipt

Expected Result:

Should be showing from the top of the receipt

Actual Result:

Showing the middle portion of the receipt

Workaround:

unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

  • Android: Native
  • Android: mWeb Chrome
  • iOS: Native
  • iOS: mWeb Safari
  • MacOS: Chrome / Safari
  • MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

Screen Shot 2024-01-09 at 12 00 13 AM

View all open jobs on GitHub

Upwork Automation - Do Not Edit
  • Upwork Job URL: https://www.upwork.com/jobs/~019bb6b442f89fba02
  • Upwork Job ID: 1744585981661257728
  • Last Price Increase: 2024-04-24
  • Automatic offers:
    • cubuspl42 | Reviewer | 28117185
    • dukenv0307 | Contributor | 28117186

Activity

  1. added
    ExternalAdded to denote the issue can be worked on by a contributor
    BugSomething is broken. Auto assigns a BugZero manager.
    on Jan 9, 2024
  2. changed the title [-][Wave 5] Receipt thumbnail should show the top of the receipt[/-] [+][$500] [Wave 5] Receipt thumbnail should show the top of the receipt[/+] on Jan 9, 2024
  3. melvin-bot commented on Jan 9, 2024

    @melvin-bot
  4. melvin-bot commented on Jan 9, 2024

    @melvin-bot

    Triggered auto assignment to @adelekennedy (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

  5. added
    Help WantedApply this label when an issue is open to proposals by contributors
    on Jan 9, 2024
  6. melvin-bot commented on Jan 9, 2024

    @melvin-bot

    Triggered auto assignment to Contributor-plus team member for initial proposal review - @cubuspl42 (External)

  7. dukenv0307 commented on Jan 9, 2024

    @dukenv0307
    Contributor

    Proposal

    Please re-state the problem that we are trying to solve in this issue.

    Showing the middle portion of the receipt

    What is the root cause of that problem?

    For resizeMode cover, the image will have default positioning which is in the middle. The behavior that we want is similar to the object-position: top in CSS, but this React Native doesn't support it out of the box so we need to add it to React Native.

    What changes do you think we should make in order to solve the problem?

    Make the View wrapping the image to have overflow hidden, for the image itself, it will take full width, variable height according to the aspectRatio (not fixed height).

    The View wrapper already has overflow hidden here so all we have to do is to make sure our Image component can handle the object position top case.

    1. Add a objectPositionTop boolean prop to our Image component to control this behavior (later we can modify to support other positions like bottom). I intended to make it as similar and reusable as possible (similar usage as native object-position: top in CSS), we just need to modify in common Image component and we can use it everywhere we use images in the app.
    2. Add the prop to ImageWithSizeCalculation as well and pass it through to the inner Image, In here, add objectPositionTop because we want that behavior for ThumbnailImage. We can make it a prop of ThumbnailImage as well so we can control the position of ThumbnailImage.
    3. Then handle in Image so the objectPositionTop property works
      Here
    const [aspectRatio, setAspectRatio] = useState();
    

    Here

    RNImage.getSize(source.uri, (width, height) => {
        onLoad({nativeEvent: {width, height}});
    
        if (props.objectPositionTop) {
            setAspectRatio(height ? width / height : 'auto');
        }
    });
    

    Here

    style={[forwardedProps.style, aspectRatio !== undefined && {aspectRatio, height: 'auto'}]}
    

    For native platforms, we should do similarly in the index.native file, for native platforms, we won't get the image dimensions unless we load the image, we might want to use Image.getSize similar to on web to make sure it's independent from the image loading, and then we can prevent displaying of the image while the aspectRatio is not yet available, it might help prevent some potential issues.

    We need to double check all possible cases of showing thumbnail receipt and add objectPositionTop to those places as needed.

    What alternative solutions did you explore? (Optional)

    Another approach is to translate the image vertically an amount such that instead of appearing from the center, image will appear at the top, this will require some math to calculate the translation so I prefer the aspectRatio approach above

    I explored using object-position: top and background-position: top as well but react native does not yet support it.

  8. aswin-s commented on Jan 9, 2024

    @aswin-s
    Contributor

    Proposal

    Please re-state the problem that we are trying to solve in this issue.

    Receipt thumbnail is not showing the top of the image

    What is the root cause of that problem?

    For displaying a receipt <ThumbnailImage/> component is used with shouldDynamicallyResize prop set to false. This causes the image to fill to the given container width and height. However the ImageWithSizeCalculation component covers the image to available width and height. This ensures that the displayed image properly covers the parent container. However a side effect of cover is that the image gets scaled and positioned such that it entirely covers the container, there by causing the vertical or horizontal alignment of image to be adjusted as needed.

    What changes do you think we should make in order to solve the problem?

    For always displaying the top section of receipt thumbnail, the image container width needs to be set to the width of thumbnail and the height should be set as per aspect ratio of the image. This makes sure that the thumbnail image gets rendered fully and not get scaled or repositioned due to 'cover'.

    function ThumbnailImage({previewSourceURL, style, isAuthTokenRequired, imageWidth = 200, imageHeight = 200, shouldDynamicallyResize = true}: ThumbnailImageProps) {
        // Use a variable to save container width on layout
        const [containerSize, setContainerSize] = useState({ width: 0, height: 0 });
        ...
    
        // Now compute the aspect ratio of the image
        const aspectRatio = currentImageWidth && currentImageHeight ? currentImageWidth / currentImageHeight : 1;
        // Compute aspect ratio of container
        const containerAspectRatio = containerSize.width / containerSize.height;
        
        // Get static image size as per aspect ratios
        const getImageSize = useCallback(() => {
            if (!aspectRatio) {
                return [styles.w100, styles.h100]
            }
            if (aspectRatio > containerAspectRatio) {
                return [{
                    height: containerSize.height,
                    width: containerSize.height * aspectRatio
                }]
            }
            return [{
                width: containerSize.width,
                height: containerSize.width / aspectRatio
            }]
        }, [aspectRatio, containerAspectRatio, containerSize.height, containerSize.width, styles.h100, styles.w100])
    
        const sizeStyles = shouldDynamicallyResize ? [StyleUtils.getWidthAndHeightStyle(currentImageWidth ?? 0, currentImageHeight)] : getImageSize();
     
        return (
            <View
                style={[style, styles.overflowHidden]}
    	    // Get container width             
                onLayout={(e) => setContainerSize(e.nativeEvent.layout)}
            >
                <View style={[...sizeStyles, styles.alignItemsCenter, styles.justifyContentCenter]}>
                    <ImageWithSizeCalculation
                        url={previewSourceURL}
                        onMeasure={updateImageSize}
                        isAuthTokenRequired={isAuthTokenRequired}
                    />
                </View>
            </View>
        );

    Above logic also handles images whose width is larger than height. Also the logic works for both native and web platforms.

    There are a couple of screens like MoneyRequestConfirmationList and ReportActionItemImage etc where Image component is used instead of ThumbnailImage for displaying receipt. Such instances need to be replaced so that all occurrences of receipt image is properly displayed

    Result

    iOS

    fix-ios_out.mp4

    Web

    fix-web_out.mp4

    What alternative solutions did you explore? (Optional)

    None (edited)

  9. dukenv0307 commented on Jan 9, 2024

    @dukenv0307
    Contributor

    @aswin-s FYI your solution's the same as one of my alternate solution

    A potential alternative is to make the View wrapping the image to have overflow hidden, for the image itself, it will take full width, variable height according to the aspectRatio (not fixed height).

    I'm writing the implementation details for my proposal and updating soon

  10. shubham1206agra commented on Jan 9, 2024

    @shubham1206agra
    Contributor

    Proposal

    Please re-state the problem that we are trying to solve in this issue.

    Receipt thumbnail is not showing the top of the image

    What is the root cause of that problem?

    Inside ReportActionItemImage, the ThumbnailImage component is being used. But this component uses 100% for height and width, which makes image center (looks like center but not exactly).

    What changes do you think we should make in order to solve the problem?

    We need to use aspectRatio technique to properly scale the images so that always top can be shown. We need to correctly scale the images for landscape, and portrait images.

    Test branch - https://github.com/shubham1206agra/App/tree/test-thumbnail

    Note - Naming of function is not final.

    What alternative solutions did you explore? (Optional)

    NA

  11. aswin-s commented on Jan 9, 2024

    @aswin-s
    Contributor

    Noted @dukenv0307. My proposal has just one approach detailed which works on all platforms. I've tested on both iOS and web and shared the screen grabs as well. I've also addressed scenarios like serving local image when receipt upload is in progress in the above video. So it is not exactly same or abstract like your proposal.

    I'll leave the merit of the proposal to the C+.

  12. 116 remaining items

  13. Santhosh-Sellavel commented on May 2, 2024

    @Santhosh-Sellavel
    Collaborator

    @mallenexpensify/@ad Can you assign me here please, seems I got unassigned accidentally

    @Santhosh-Sellavel reassigning, please take over as C+. If you don't have bandwidth, unassign yourself.

  14. mallenexpensify commented on May 3, 2024

    @mallenexpensify
    Contributor

    Done, added @Santhosh-Sellavel , sorry about that

  15. melvin-bot commented on May 10, 2024

    @melvin-bot

    @Beamanator, @adelekennedy, @Santhosh-Sellavel, @dukenv0307 Whoops! This issue is 2 days overdue. Let's get this updated quick!

  16. melvin-bot commented on May 14, 2024

    @melvin-bot

    @Beamanator, @adelekennedy, @Santhosh-Sellavel, @dukenv0307 Still overdue 6 days?! Let's take care of this!

  17. Beamanator commented on May 14, 2024

    @Beamanator
    Contributor

    Hmm @dukenv0307 your PR went to prod a week or so ago, are we waiting on anything here?

  18. dukenv0307 commented on May 15, 2024

    @dukenv0307
    Contributor

    @Beamanator We're waiting for payment here.

  19. added
    Awaiting PaymentAuto-added when associated PR is deployed to production
    and removed
    ReviewingHas a PR in review
    on May 19, 2024
  20. mallenexpensify commented on May 19, 2024

    @mallenexpensify
    Contributor

    Looks like this hit production on 4/23, so gonna pay
    #35041 (comment)

    Contributor: @dukenv0307 paid $250 via Upwork
    Contributor+: @Santhosh-Sellavel owed $250 via NewDot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Awaiting PaymentAuto-added when associated PR is deployed to productionBugSomething is broken. Auto assigns a BugZero manager.DailyKSv2ExternalAdded to denote the issue can be worked on by a contributor

Type

No type

Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions