🍱 Interactive widgets

[juliusknorr]

Summary

Goal

Current link previews provide a simple, static preview but in lots of contexts it would be very helpful if users could work more interactively with the linked resource without navigating away to the actual app of it.

This is partly addressing a gap to competitor products like Notion, Microsoft Loop, Sharepoint when on the Collectives app, but also in other Nextcloud parts like Talk where team/discord can display more interactive widgets within the chat.

The goal is to add an additional way of rendering link preview widgets that provide more information, interactivity and possibly useful actions right away. The degree of what functions should be available for a widget may need further discussion on the individual apps.

Scope

• We provide a generic way that existing link preview widgets custom render function can be extended to provide an interactive way Add interactive mode for widget registration and rendering nextcloud-libraries/nextcloud-vue#5090
• Using the interactive widget
  • Text/collectives will integrate using this interactive preview mode if manually inserted by the user Nodes for embedding / previewing content text#5270
  • Deck/notes will inherit this function from the text app automatically
• Providing an interactive widget
  • As per discussion with Jan we considered the following interactive widgets as the MVP, but need more specific design specs individually
    • Files/folder (e.g. pdf preview embedding)
    • calendar for shared calendars with month/day/week picker, single events 🍱 Interactive widget for calendar events calendar#5701
    • deck full board, single card
    • tables full table exists, single rows, fully interactive Interactive preview support tables#803
    • youtube
    • spotify
    • maps
    • github

Out of scope

• Mobile app support

Tracking issues

• Add interactive mode for widget registration and rendering nextcloud-libraries/nextcloud-vue#5090
• Nodes for embedding / previewing content text#5270
• 🍱 Interactive widget for calendars calendar#5746
• 🍱 Interactive widget: file #42916
• 🍱 Interactive widget: folder #42917
• Interactive preview support tables#803
• Interactive board widget deck#5630

Follow-ups

Not planned at the moment

• 🍱 Interactive widget for emails mail#9253
• 🍱 Interactive widget for calendar events calendar#5701
• 🍱 Interactive widget contacts#3790

## More detailed draft notes

• Deeper embedding of resources old mockup
• Add embedding (smart picker like) but not as link preview but real interactive widget
• Should be based on the existing smart picker concept of inserting links
• Should work across apps
  • For link preview providers (define what apps should implement this)
  • For displaying (e.g. talk or where else the link preview is used)
• To clarify
  • How does the user or app decide if a simple or interactive preview is shown
  • How to handle this on mobile
  • Text/Collectives/Notes
    • Should this be different markdown syntax so users can decide based on the document usage what to display?
    • We got feedback that the link preview is sometimes not useful in its current approach of always rendering it if it is the only content of a paragraph
• Reference: Tables already has an option like that to select the mode, but we should probably have a generic approach
  

Notion examples:

Design spec

I had a call with @jancborchardt to outline the implementation in more detail:

• We should have different modes of link previews / widget renderings similar to Notion
  • Inline link (auto generate title and just show without a link preview)
  • Static link preview from open graph
  • Interactive widget if it makes sense
  • When inserting a link users should be able to select which option is used
    • default, preference should be depending on the link type but remember user preference
• Should have a global setting for privacy opt out
• Interactive link previews either show a collection or single item
  • Limit scope of interactivity to browsing, but some functions like create new entries still make sense
  • There need to be reasonable default sizing of widgets
• Apps (bold as important/prio items)
  • files/folder
    • pdf preview embedding
    • possibly readonly for odt
  • photo/album
  • talk room/message
  • mail no preview 🍱 Interactive widget for emails mail#9253
  • contact maybe for shared addressbook 🍱 Interactive widget contacts#3790
  • calendar for shared calendars with month/day/week picker, single events 🍱 Interactive widget for calendar events calendar#5701
    • single events like calendar quick view
    • full calendar embed
  • notes single note, possible category
  • collectives just simple preview
  • deck full board, single card
  • tasks single task, full task list
  • forms complete form, form results (interesting)
  • tables full table exists, single rows, fully interactive
  • external apps
    • youtube already embedded
    • spotify can be embedded
    • maps can be embedded
    • draw io
    • sketch picker
    • github/jira/openproject interactive list depending on the apps

Implementation questions

• Figure out how we can store different link preview formats in markdown for text @max-nextcloud
• For displaying different link formats it could be beneficial to switch to a node instead of a mark
  • I did a quick PoC at https://github.com/nextcloud/text/compare/feat/link-node also covering auto generating the link title from link preview metadata
• Link previews/smart picker @juliushaertl
  • Need a way to indicate that a widget or link preview supports interactive widgets
  • Rendering the widget needs to let integrating developers decide which one to show
• Might be good to have Configurable line-length for Editor text#4024 for this

[jancborchardt]

Btw this is a bit of an advanced and fancy example of how Microsoft does it with their Loop components (thanks @marcoambrosini for sharing). Fully collaborative, including live cursor and avatars on top:

00.StayInSyncHd_video_en-us-0x1080-6439k.mp4

[juliusknorr]

Discussion with @max-nextcloud around markdown syntax:

We need three options:

inline link

[]()
[http://](http://)

regular preview

plain url as the only thing within a paragraph

https://nextcloud.com

interactive rendering

<https://nextcloud.com>

other options:

#<http>

@[max](mention://)


[test](https://example.com)<!-- -->


```preview

```

:::error
https://example.com
:::

[juliusknorr]

First draft for adding a flag for this to the reference rendering API https://github.com/nextcloud-libraries/nextcloud-vue/compare/feat/noid/interactive-widget

Tables app implementing it: https://github.com/nextcloud/tables/pull/new/feat/noid/interactive-widget

Still to figure out:

• Do we need size recommendations, restrictions (might be good if talk can enforce a maximum height but collectives could just render depending on required height)
• Do we need a generic way to pick and return different types?