Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
texta
texta-mlp-python
Commits
80179254
Commit
80179254
authored
Sep 01, 2021
by
Marko Kollo
😄
Browse files
"Fix" for processing list values.
parent
f28545be
Pipeline
#6259
passed with stage
in 9 minutes and 41 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
texta_mlp/mlp.py
View file @
80179254
...
...
@@ -10,12 +10,11 @@ import regex as re
import
stanza
from
bs4
import
BeautifulSoup
from
langdetect
import
detect
from
pelecanus
import
PelicanJson
from
texta_mlp.document
import
Document
from
texta_mlp.entity_mapper
import
EntityMapper
from
texta_mlp.utils
import
parse_bool_env
from
texta_mlp.exceptions
import
LanguageNotSupported
from
texta_mlp.utils
import
parse_bool_env
# Languages supported by default.
...
...
@@ -354,7 +353,7 @@ class MLP:
lang_group
=
{}
texts
=
[
Document
.
parse_doc
(
doc_path
,
document
)
for
document
in
docs
]
for
index
,
text
in
enumerate
(
texts
):
text
=
text
[
0
]
if
text
and
len
(
text
)
==
1
else
text
text
=
text
[
0
]
if
text
else
""
lang
=
self
.
detect_language
(
text
)
if
lang
not
in
self
.
supported_langs
:
lang
=
self
.
default_lang
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment