Repository navigation
Bash driver is not a detected language #39
Description
Activity
cc @mcuadros what should we do about this? I don't think enry (or github/linguist) is ever going to be able to tell bash apart other shell dialects in a reliable way. Should we just document this and recommend users to use
Bashlanguage when enry outputsShell? This would be pretty nasty in gitbase...support more than one language name per driver in the manifest
I like this idea as it simplifies things for clients as they can just call bblfsh with output of enry
Reacted by Santiago M. MolaSmall summary of the context for this issue below.
Use case
- client asks bblfshd for list of the supported languages
- client uses enry to detect language and if it's supported
- client asks bblfshd to parse \w a given language
Problem
- "Language" field from the driver manifest can be different from language identifier that Enry uses.
Example: bash (manifest) VS shell (enry)
Alternatives
- 🙅♂️ rename driver to
shell-driver. Discarded, see Some lang names guessed by bblfsh has a different driver name bblfshd#162 (comment) - change
Languagefield to support multiple drivers (either as array, or as a string \w separator)- "language" field is used in many places for different purposes
- changes would be invasive and include code for new driver generation, etc, etc
- add new field to manifest
"enry_languages""language_aliase"- use it to map incoming "language" from parse requests, before DriverPool selection
- depending on whether we want to impose changes on clients or not:
- either just add new field to v1
protocol1.DriverManifestand expect clients to update client version and read it instead - or reply to
protocol1.SupportedLanguageswith additional "fake"DriverManifest, one per each item of"enry_languages""language_aliase"arrays.
- either just add new field to v1
It may be a good time to implement any of this directly in v2, when we decide to port
SupportedLanguagesRequestfrom protocol v1 to v2.WDYT about approach 3? Feedback is very welcome.
My proposal (make the languages field a list or a string with a separator) doesn't imply generating multiple drivers, just multiple names for the same drivers.
My proposal (make the languages field a list or a string with a separator) doesn't imply generating multiple drivers, just multiple names for the same drivers.
I'm not familiar with the codebase very much, but a quick glance showed that in this case, on top of the mandatory changes listed in 3, we would also need to
- update the protocol definition (if changing a field type)
- introduce new special logic in driver generation, as "language" field is used all over templates
Would not that be the case? If you could give some preliminary pointers to which parts of the codebase would be touched by such proposal, that would be very appreciated!
Yes, changes are going to be needed in any case. We could also keep the same language field and all a "language_aliases" field (instead of "enry_languages" so we're not tying the drivers to enry in any way).
I would rather use something like
language_aliasesto avoid tying the protocol to enry itself. If we really wanted this to reference enry, I would reference linguist instead (linguist_languages), since that is our ground truth for language names.Reacted by Alex@juanjux @smola
language_aliasesthank you guys, makes perfect sense 👍 Updated the msg above.Having a separate field have benefits that at least we do not need to touch any templates/driver generation part. So far, the only changes that I found to be needed in this case seem to be well localized and are listed in 3. (And we even seem to have an option of not changing a protocol1 at all! I vote for that.)
If there is no more feedback, I will be moving result of this discussion with a proposal to a separate issue in SDK in next few days, as it's not Bash specific at all.
We can keep this one open until that one is merged and released.
I'm fine with the new field then, too.
- added a commit that references this issue
on May 24, 2019 - added a commit that references this issue
on May 28, 2019
We use enry for language detection both on bblfshd side as well as on our applications. Enry, as well as github/linguist, do not detect Bash itself, but just Shell (this includes the whole family of POSIX-like shell languages).
This generates the following problem: