[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disallow string input when tokenizer is set to None #12

Closed
LSYS opened this issue Nov 21, 2021 · 0 comments · Fixed by #38
Closed

Disallow string input when tokenizer is set to None #12

LSYS opened this issue Nov 21, 2021 · 0 comments · Fixed by #38
Labels
enhancement New feature or request

Comments

@LSYS
Copy link
Owner
LSYS commented Nov 21, 2021

For custom NLP pipelines, disallow string input when tokenizer is set to None.

Assert that input is a list of words instead of a string.
@LSYS LSYS added the enhancement New feature or request label Nov 21, 2021
LSYS added a commit that referenced this issue Aug 20, 2022
@LSYS
Disallow string input if =None (closes #12)
3d737d3 
- Removing the gotcha where string inputs can still be accepted into the LexicalRichness class when  is set to None. Input should be a list of strings. Closes #12
@LSYS LSYS mentioned this issue Aug 20, 2022
Merged
@LSYS LSYS closed this as completed in #38 Aug 20, 2022
LSYS added a commit that referenced this issue Aug 20, 2022
@LSYS
Merge pull request #38 from LSYS/patch
da38e74 
Disallow string input if `tokenizer` =None (closes #12)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Disallow string input if tokenizer =None (closes #12) LSYS/LexicalRichness
1 participant
@LSYS