[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement multi-lingual message and signal descriptions #240

Merged
merged 2 commits into from
Dec 1, 2020

Conversation

andlaus
Copy link
Member
@andlaus andlaus commented Nov 30, 2020

This PR continues the quest for ARXML-3 support, cf https://github.com/andlaus/cantools/tree/arxml_refactor

ATTENTION: This patch includes some changes outside of the ARXML loader.

For now, the default is to assume that comments which don't explicitly specify a language are in English. An alternative would be to check if the dictionary containing the comments contains exactly a single element and use that instead. The problem with this approach is that comments for different languages might get mixed up more easily: Assume a supplier based in China who documents its signals only in Chinese and a OEM based in Brazil who likes to document its stuff using Portuguese...

Andreas Lauser andreas.lauser@mbition.io, Mercedes-Benz AG on behalf of MBition GmbH.

Imprint

@coveralls
Copy link
coveralls commented Nov 30, 2020

Coverage Status

Coverage decreased (-0.4%) to 96.517% when pulling 3ae062d on Daimler:multilang_descriptions into 173ba03 on eerimoq:master.

@andlaus
Copy link
Member Author
andlaus commented Nov 30, 2020

the test coverage should include the missing lines once the ARXML-3 patch is in. I will add it to the ARXML-4 files if you want, but this would slightly delay things for the rest of this series...

Copy link
Collaborator
@eerimoq eerimoq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know what you think about my suggestions.

cantools/database/can/message.py Outdated Show resolved Hide resolved
cantools/database/can/message.py Outdated Show resolved Hide resolved
cantools/database/can/message.py Outdated Show resolved Hide resolved
cantools/database/can/message.py Outdated Show resolved Hide resolved
@andlaus andlaus force-pushed the multilang_descriptions branch 2 times, most recently from 9d9052c to b823de7 Compare November 30, 2020 12:41
@andlaus
Copy link
Member Author
andlaus commented Nov 30, 2020

the last revions had some serious issues, sorry for that. The new one fixes this.

@andlaus andlaus force-pushed the multilang_descriptions branch 2 times, most recently from bacf8b8 to de61daa Compare November 30, 2020 14:01
note that for now the default is to assume that comments which don't
explicitly specify a language are in English. An alternative would be
to check if the dictionary containing the comments contains exactly a
single element and use that instead. The problem with this approach is
that comments for different languages might get mixed up more easily:
Assume a supplier based in China who documents its signals only in
Chinese and a OEM based in Brazil who likes to document its stuff
using Portuguese...
They where oversights made due to splitting up the big branch that
adds AUTOSAR 3 support into a bunch of smaller and easy to review pull
requests.
@eerimoq
Copy link
Collaborator
eerimoq commented Dec 1, 2020

It's not obvious to me how this should be designed. Suppose you have an ARXML-file with comments in german (no english) which you want to convert to a DBC-file. With the logic in the PR the DBC-file would not contain any comments as only None and EN are checked when using .comment (which the DBC dump function does). I would expect the DBC-file to have comments in german. Converting from DBC to ARXML (which is not currently supported) would have similar problems.

One alternative design is to add a language argument to all classes and functions that need to know the language in use. By default it would probably be english (EN). All file formats without language information would gladly accept the language choice. Only file formats like ARXML would actually use the language. This would allow loading a DBC-file with comments in german to generate an ARXML with "DE" comments (given that the user passes language='DE' when loading the DBC-file).

.comment is in the selected language. .comments is a dictionary of comments in all available languages.

@andlaus
Copy link
Member Author
andlaus commented Dec 1, 2020

I suppose that the root of this problem is that the capabiities of DBC file format and ARXML are not identical for our purposes. I agree that converting all internal code to always use a dictionary when it comes to comments would do the trick, as far as I can see. For the ARXML-to-DBC converter, I also see this the same as you: One could have specify the language which ought to be used in the DBC externally and let the .comment attribute default to something reasonable (e.g., 'EN' if multiple languages are defined, and the only element in the dictionary if the dictionary size is one)? The question then becomes: What's a good default policy?

@andlaus
Copy link
Member Author
andlaus commented Dec 1, 2020

BTW: Do you happen know why coveralls again claims that the overall coverage is down, but if you inspect the results, coverage for all individual files is equal or increased? (maybe I just misinterpret its results)

@eerimoq
Copy link
Collaborator
eerimoq commented Dec 1, 2020

No, I do not know why the coverage is down again =)

We don't need any default policy, the language is known with Message(..., language='EN') if we go for that design. language='EN' has to be added in lots of functions and classes, but that should be fine.

@andlaus
Copy link
Member Author
andlaus commented Dec 1, 2020

ok, let's do it this way then. later?

@eerimoq
Copy link
Collaborator
eerimoq commented Dec 1, 2020

Sure, let's postpone it if it makes things easier for you.

@andlaus
Copy link
Member Author
andlaus commented Dec 1, 2020

could you please press The Green Button? (it probably is grey ATM, though ;)

@eerimoq eerimoq merged commit 32bc872 into cantools:master Dec 1, 2020
@andlaus
Copy link
Member Author
andlaus commented Dec 1, 2020

thanks :)

@andlaus andlaus deleted the multilang_descriptions branch December 1, 2020 12:43
@eerimoq
Copy link
Collaborator
eerimoq commented Dec 1, 2020

I actually meant to postpone the whole PR, but let's just redesign it later (probably before a new release).

@andlaus
Copy link
Member Author
andlaus commented Dec 1, 2020

I actually meant to postpone the whole PR, but let's just redesign it later (probably before a new release).

+1. For my current goal (i.e., proper support for Daimler-style ARXML-3 CAN description files), this is just a side concern, though. That said, I'll help with proper internationalization support where I can.

@eerimoq
Copy link
Collaborator
eerimoq commented Dec 1, 2020

What's special with Daimler ARXML-files? Will other companies/users still be able to use cantools? I certainly hope so.

@andlaus
Copy link
Member Author
andlaus commented Dec 1, 2020

What's special with Daimler ARXML-files?

not much. They are basically just standard ARXML-3 files but they use a lot of features, and use a weird XML namespace.

So far all of my PRs equally applied to ARXML-4...

@andlaus
Copy link
Member Author
andlaus commented Dec 1, 2020

i.e., xmllint says they validate against the AUTOSAR 3.2 XSD which I found in the internet after changing the namespace.

@eerimoq
Copy link
Collaborator
eerimoq commented Dec 1, 2020

Great, you certainly add much value to the ARXML module, and if it's close enough to the standard it should be easy to support the general case as well if someone asks for it/implements it.

VonSquiggles pushed a commit to VonSquiggles/cantools that referenced this pull request May 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants