pool/python-charset-normalizer
SHA256
20
0
Fork 2
Code Issues Pull Requests Activity

Commit Graph

  • 7e8d58bccf Accepting request 1344812 from devel:languages:python factory dimstar_suse 2026-04-08 15:13:29 +00:00
  • b55fcd72a9 - update to 3.4.7: * Pre-built optimized version using mypy[c] v1.20. * Relax setuptools constraint to setuptools>=68,<82.1. * Correctly remove SIG remnant in utf-7 decoded string. (#718) dirkmueller 2026-04-06 21:44:31 +00:00
  • e21b5a19ec Accepting request 1343405 from devel:languages:python anag_factory 2026-03-30 16:29:45 +00:00
  • f7d33c691b - update to 3.4.6: * Flattened the logic in charset_normalizer.md for higher performance. Removed eligible(..) and feed(...) in favor of feed_info(...). * Updated UNICODE_RANGES_COMBINED using Unicode blocks v17. * Edge case where noise difference between two candidates can be almost insignificant. * CLI --normalize writing to wrong path when passing multiple files in. * Freethreaded pre-built wheels now shipped in PyPI starting with 3.14t. * Update setuptools constraint to setuptools>=68,<=82. * Raised upper bound of mypyc for the optional pre-built extension to v1.19.1 * Add explicit link to lib math in our optimized build. * Logger level not restored correctly for empty byte sequences. * TypeError when passing bytearray to from_bytes. * Applied safe micro-optimizations in both our noise detector and language detector. * Rewrote the query_yes_no function (inside CLI) to avoid using ambiguous licensed code. * Added cd.py submodule into mypyc optional compilation to reduce further the performance impact. dirkmueller 2026-03-29 10:16:17 +00:00
  • 6b4533885b Accepting request 1315251 from devel:languages:python anag_factory 2025-11-05 15:17:48 +00:00
  • ce5ef5f0c0 Accepting request 1314564 from home:glaubitz:branches:devel:languages:python dgarcia 2025-11-03 09:19:03 +00:00
  • 7e2ef73dfc Accepting request 1304688 from devel:languages:python anag_factory 2025-09-15 17:50:31 +00:00
  • 91d56d242a - update to 3.4.3: * mypy(c) is no longer a required dependency at build time if CHARSET_NORMALIZER_USE_MYPYC isn't set to 1. (#595) * automatically lower confidence on small bytes samples that are not Unicode in detect output legacy function. * Custom build backend to overcome inability to mark mypy as an optional dependency in the build phase. * Support for Python 3.14 * sdist archive contained useless directories. * automatically fallback on valid UTF-16 or UTF-32 even if the md says it's noisy. * SBOM are automatically published to the relevant GitHub release to comply with regulatory changes. * Each published wheel comes with its SBOM. We choose CycloneDX as the format. * Prebuilt optimized wheel are no longer distributed by default for CPython 3.7 due to a change in cibuildwheel. dirkmueller 2025-09-14 21:00:37 +00:00
  • f7e46571b3 Accepting request 1274387 from devel:languages:python slfo-main slfo-1.2 anag_factory 2025-05-06 14:38:59 +00:00
  • d9b2ef1a60 Accepting request 1274387 from devel:languages:python anag_factory 2025-05-06 14:38:59 +00:00
  • 35295358a7 - Update to 3.4.2 * Addressed the DeprecationWarning in our CLI regarding argparse.FileType by backporting the target class into the package. (#591) * Improved the overall reliability of the detector with CJK Ideographs. (#605) (#587) * Optional mypyc compilation upgraded to version 1.15 for Python >= 3.8 mcalabkova 2025-05-05 09:15:39 +00:00
  • 940d389e13 Accepting request 1274349 from home:glaubitz:branches:devel:languages:python mcalabkova 2025-05-05 09:15:39 +00:00
  • 93a74fa152 Accepting request 1238022 from devel:languages:python anag_factory 2025-01-16 17:31:17 +00:00
  • 2cb57d39c4 Accepting request 1238022 from devel:languages:python anag_factory 2025-01-16 17:31:17 +00:00
  • b17aad8053 - Use libalternatives instead of update-alternatives, bsc#1235781 dgarcia 2025-01-15 10:38:37 +00:00
  • ceff5b28c1 - Use libalternatives instead of update-alternatives, bsc#1235781 dgarcia 2025-01-15 10:38:37 +00:00
  • 06686c6c46 Accepting request 1236172 from devel:languages:python anag_factory 2025-01-12 10:10:17 +00:00
  • a683fc34d0 Accepting request 1236172 from devel:languages:python anag_factory 2025-01-12 10:10:17 +00:00
  • 7e521f133b - Update to 3.4.1 * Project metadata are now stored using pyproject.toml instead of setup.cfg using setuptools as the build backend. * Enforce annotation delayed loading for a simpler and consistent types in the project. * Optional mypyc compilation upgraded to version 1.14 for Python >= 3.8 * Added pre-commit configuration. * Added noxfile. * Removed build-requirements.txt as per using pyproject.toml native build configuration. * Removed bin/integration.py and bin/serve.py in favor of downstream integration test (see noxfile). * Removed setup.cfg in favor of pyproject.toml metadata configuration. * Removed unused utils.range_scan function. * Converting content to Unicode bytes may insert utf_8 instead of preferred utf-8. (#572) * Deprecation warning "'count' is passed as positional argument" when converting to Unicode bytes on Python 3.13+ - Drop sed command to remove code coverage flags from pytest mcalabkova 2025-01-09 12:21:13 +00:00
  • 561398d356 Accepting request 1236124 from home:glaubitz:branches:devel:languages:python mcalabkova 2025-01-09 12:21:13 +00:00
  • f142f583d0 Accepting request 1221058 from devel:languages:python anag_factory 2024-11-05 14:39:43 +00:00
  • 0ee7e60e59 Accepting request 1221058 from devel:languages:python anag_factory 2024-11-05 14:39:43 +00:00
  • 8e7f63d1ba - switch to PEP517 build dirkmueller 2024-10-28 16:38:02 +00:00
  • ce57742542 - switch to PEP517 build dirkmueller 2024-10-28 16:38:02 +00:00
  • e1a6b9e55c Accepting request 1217078 from devel:languages:python anag_factory 2024-10-23 19:08:21 +00:00
  • 2535bce82a Accepting request 1217078 from devel:languages:python anag_factory 2024-10-23 19:08:21 +00:00
  • 3bf31d75b8 - update to 3.4.0: * Argument --no-preemptive in the CLI to prevent the detector to search for hints. * Support for Python 3.13 * Relax the TypeError exception thrown when trying to compare a CharsetMatch with anything else than a CharsetMatch. * Improved the general reliability of the detector based on user feedbacks. (#520) (#509) (#498) (#407) * Declared charset in content (preemptive detection) not changed when converting to utf-8 bytes. dirkmueller 2024-10-22 16:00:24 +00:00
  • 98a4dbb463 - update to 3.4.0: * Argument --no-preemptive in the CLI to prevent the detector to search for hints. * Support for Python 3.13 * Relax the TypeError exception thrown when trying to compare a CharsetMatch with anything else than a CharsetMatch. * Improved the general reliability of the detector based on user feedbacks. (#520) (#509) (#498) (#407) * Declared charset in content (preemptive detection) not changed when converting to utf-8 bytes. dirkmueller 2024-10-22 16:00:24 +00:00
  • 223430a4ca Accepting request 1128743 from devel:languages:python anag_factory 2023-11-27 21:42:20 +00:00
  • 39f28f6f2d Accepting request 1128743 from devel:languages:python anag_factory 2023-11-27 21:42:20 +00:00
  • 9cd0e22679 - update to 3.3.2: * Unintentional memory usage regression when using large payload that match several encoding (#376) * Regression on some detection case showcased in the documentation (#371) * Noise (md) probe that identify malformed arabic representation due to the presence of letters in isolated form * Optional mypyc compilation upgraded to version 1.6.1 for Python >= 3.8 * Improved the general detection reliability based on reports from the community dirkmueller 2023-11-25 14:12:46 +00:00
  • 4d62239b35 - update to 3.3.2: * Unintentional memory usage regression when using large payload that match several encoding (#376) * Regression on some detection case showcased in the documentation (#371) * Noise (md) probe that identify malformed arabic representation due to the presence of letters in isolated form * Optional mypyc compilation upgraded to version 1.6.1 for Python >= 3.8 * Improved the general detection reliability based on reports from the community dirkmueller 2023-11-25 14:12:46 +00:00
  • 5a4b0b3e0c Accepting request 1114778 from devel:languages:python anag_factory 2023-11-23 20:38:43 +00:00
  • 2c41d8af7f Accepting request 1114778 from devel:languages:python anag_factory 2023-11-23 20:38:43 +00:00
  • 3e7e8a34ba - update to 3.3.0: * Allow to execute the CLI (e.g. normalizer) through python -m charset_normalizer.cli or python -m charset_normalizer * Support for 9 forgotten encoding that are supported by Python but unlisted in encoding.aliases as they have no alias * Optional mypyc compilation upgraded to version 1.5.1 for Python >= 3.7 * Unable to properly sort CharsetMatch when both chaos/noise and coherence were close due to an unreachable condition in \_\_lt\_\_ (#350) - Update to 3.0.1 - Update to 3.0.0 * ASCII miss-detection on rare cases (PR #170) * Wrong logging level applied when setting kwarg explain to True - require lower-case name instead of breaking build dirkmueller 2023-10-02 09:08:45 +00:00
  • b5ba82d155 - update to 3.3.0: * Allow to execute the CLI (e.g. normalizer) through python -m charset_normalizer.cli or python -m charset_normalizer * Support for 9 forgotten encoding that are supported by Python but unlisted in encoding.aliases as they have no alias * Optional mypyc compilation upgraded to version 1.5.1 for Python >= 3.7 * Unable to properly sort CharsetMatch when both chaos/noise and coherence were close due to an unreachable condition in \_\_lt\_\_ (#350) - Update to 3.0.1 - Update to 3.0.0 * ASCII miss-detection on rare cases (PR #170) * Wrong logging level applied when setting kwarg explain to True - require lower-case name instead of breaking build dirkmueller 2023-10-02 09:08:45 +00:00
  • efcb074653 Accepting request 1098807 from devel:languages:python anag_factory 2023-07-17 17:22:47 +00:00
  • 69d02200cb Accepting request 1098807 from devel:languages:python anag_factory 2023-07-17 17:22:47 +00:00
  • c103080fc4 - update to 3.2.0: * Typehint for function from_path no longer enforce PathLike as its first argument * Minor improvement over the global detection reliability * Introduce function is_binary that relies on main capabilities, and optimized to detect binaries * Propagate enable_fallback argument throughout from_bytes, from_path, and from_fp that allow a deeper control over the detection (default True) * Edge case detection failure where a file would contain 'very- long' camel cased word (Issue #289) dirkmueller 2023-07-11 13:24:00 +00:00
  • 2419fdf02e - update to 3.2.0: * Typehint for function from_path no longer enforce PathLike as its first argument * Minor improvement over the global detection reliability * Introduce function is_binary that relies on main capabilities, and optimized to detect binaries * Propagate enable_fallback argument throughout from_bytes, from_path, and from_fp that allow a deeper control over the detection (default True) * Edge case detection failure where a file would contain 'very- long' camel cased word (Issue #289) dirkmueller 2023-07-11 13:24:00 +00:00
  • 9ded4692d5 Accepting request 1084939 from devel:languages:python dimstar_suse 2023-05-09 11:06:32 +00:00
  • f910db3030 Accepting request 1084939 from devel:languages:python dimstar_suse 2023-05-09 11:06:32 +00:00
  • 110acf0118 - add sle15_python_module_pythons (jsc#PED-68) dirkmueller 2023-05-05 06:41:30 +00:00
  • fbcae794f8 - add sle15_python_module_pythons (jsc#PED-68) dirkmueller 2023-05-05 06:41:30 +00:00
  • e5a29dc1ed Accepting request 1074517 from devel:languages:python dimstar_suse 2023-03-29 21:26:15 +00:00
  • 5fd26c1c73 Accepting request 1074517 from devel:languages:python dimstar_suse 2023-03-29 21:26:15 +00:00
  • 5a2b102103 - update to 3.1.0: * Argument should_rename_legacy for legacy function detect and disregard any new arguments without errors (PR #262) * Removed Support for Python 3.6 (PR #260) * Optional speedup provided by mypy/c 1.0.1 dirkmueller 2023-03-26 20:04:47 +00:00
  • 41b8cd85f8 - update to 3.1.0: * Argument should_rename_legacy for legacy function detect and disregard any new arguments without errors (PR #262) * Removed Support for Python 3.6 (PR #260) * Optional speedup provided by mypy/c 1.0.1 dirkmueller 2023-03-26 20:04:47 +00:00
  • cfd5a3a805 Accepting request 1039740 from devel:languages:python dimstar_suse 2022-12-04 13:57:44 +00:00
  • 2f54e56b98 Accepting request 1039740 from devel:languages:python dimstar_suse 2022-12-04 13:57:44 +00:00
  • 20637d8d7f Accepting request 1039709 from home:yarunachalam:branches:devel:languages:python dirkmueller 2022-12-03 07:29:01 +00:00
  • 9815d6c60a Accepting request 1039709 from home:yarunachalam:branches:devel:languages:python dirkmueller 2022-12-03 07:29:01 +00:00
  • d95cdeb12d Accepting request 1032182 from devel:languages:python dimstar_suse 2022-11-04 16:31:30 +00:00
  • f4493151fe Accepting request 1032182 from devel:languages:python dimstar_suse 2022-11-04 16:31:30 +00:00
  • fa191726b9 Accepting request 1031656 from home:yarunachalam:branches:devel:languages:python mcepl 2022-10-29 11:47:59 +00:00
  • bc69402598 Accepting request 1031656 from home:yarunachalam:branches:devel:languages:python mcepl 2022-10-29 11:47:59 +00:00
  • 85cdd603a9 Accepting request 1004361 from devel:languages:python dimstar_suse 2022-09-18 15:31:58 +00:00
  • 17e5c833b9 Accepting request 1004361 from devel:languages:python dimstar_suse 2022-09-18 15:31:58 +00:00
  • 12f704616b - update to 2.1.1: * Function normalize scheduled for removal in 3.0 * Removed useless call to decode in fn is_unprintable (#206) dirkmueller 2022-09-17 15:50:18 +00:00
  • 9a8edf9b19 - update to 2.1.1: * Function normalize scheduled for removal in 3.0 * Removed useless call to decode in fn is_unprintable (#206) dirkmueller 2022-09-17 15:50:18 +00:00
  • df0a5b7224 Accepting request 998090 from devel:languages:python dimstar_suse 2022-08-20 18:27:45 +00:00
  • aa7d15bff1 Accepting request 998090 from devel:languages:python dimstar_suse 2022-08-20 18:27:45 +00:00
  • eac72ae8c5 Accepting request 998013 from home:bnavigator:branches:devel:languages:python dirkmueller 2022-08-19 06:47:38 +00:00
  • 54ae0383fc Accepting request 998013 from home:bnavigator:branches:devel:languages:python dirkmueller 2022-08-19 06:47:38 +00:00
  • 18b088feb8 Accepting request 991152 from devel:languages:python RBrownFactory 2022-07-26 17:42:09 +00:00
  • 2ea2fee032 Accepting request 991152 from devel:languages:python RBrownFactory 2022-07-26 17:42:09 +00:00
  • 8dca1a6616 - update to 2.1.0: * Output the Unicode table version when running the CLI with --version * Re-use decoded buffer for single byte character sets * Fixing some performance bottlenecks * Workaround potential bug in cpython with Zero Width No-Break Space located * in Arabic Presentation Forms-B, Unicode 1.1 not acknowledged as space * CLI default threshold aligned with the API threshold from * Support for Python 3.5 (PR #192) * Use of backport unicodedata from unicodedata2 as Python is quickly catching up, scheduled for removal in 3.0 dirkmueller 2022-07-19 11:40:33 +00:00
  • 2e435dd04d - update to 2.1.0: * Output the Unicode table version when running the CLI with --version * Re-use decoded buffer for single byte character sets * Fixing some performance bottlenecks * Workaround potential bug in cpython with Zero Width No-Break Space located * in Arabic Presentation Forms-B, Unicode 1.1 not acknowledged as space * CLI default threshold aligned with the API threshold from * Support for Python 3.5 (PR #192) * Use of backport unicodedata from unicodedata2 as Python is quickly catching up, scheduled for removal in 3.0 dirkmueller 2022-07-19 11:40:33 +00:00
  • fbde6a8151 Accepting request 954654 from devel:languages:python dimstar_suse 2022-02-16 23:29:57 +00:00
  • de8764bf5c Accepting request 954654 from devel:languages:python dimstar_suse 2022-02-16 23:29:57 +00:00
  • 259f5f1afe - update to 2.0.12: * ASCII miss-detection on rare cases (PR #170) * Explicit support for Python 3.11 (PR #164) * The logging behavior have been completely reviewed, now using only TRACE and DEBUG levels dirkmueller 2022-02-15 08:43:43 +00:00
  • 18b77e47fa - update to 2.0.12: * ASCII miss-detection on rare cases (PR #170) * Explicit support for Python 3.11 (PR #164) * The logging behavior have been completely reviewed, now using only TRACE and DEBUG levels dirkmueller 2022-02-15 08:43:43 +00:00
  • 52bba14558 Accepting request 945443 from devel:languages:python dimstar_suse 2022-01-11 20:17:16 +00:00
  • 597427c321 Accepting request 945443 from devel:languages:python dimstar_suse 2022-01-11 20:17:16 +00:00
  • c739862e1a - update to 2.0.10: * Fallback match entries might lead to UnicodeDecodeError for large bytes sequence * Skipping the language-detection (CD) on ASCII dirkmueller 2022-01-10 23:04:22 +00:00
  • 43636cbea9 - update to 2.0.10: * Fallback match entries might lead to UnicodeDecodeError for large bytes sequence * Skipping the language-detection (CD) on ASCII dirkmueller 2022-01-10 23:04:22 +00:00
  • fecde8793e Accepting request 936118 from devel:languages:python dimstar_suse 2021-12-09 18:45:18 +00:00
  • 7d3f6e53ab Accepting request 936118 from devel:languages:python dimstar_suse 2021-12-09 18:45:18 +00:00
  • 53a1bfb655 - update to 2.0.9: * Moderating the logging impact (since 2.0.8) for specific environments * Wrong logging level applied when setting kwarg explain to True dirkmueller 2021-12-06 20:09:48 +00:00
  • 4e2a28ffe3 - update to 2.0.9: * Moderating the logging impact (since 2.0.8) for specific environments * Wrong logging level applied when setting kwarg explain to True dirkmueller 2021-12-06 20:09:48 +00:00
  • 499b903d6c Accepting request 934519 from devel:languages:python dimstar_suse 2021-12-01 19:46:49 +00:00
  • b2ca28eae6 Accepting request 934519 from devel:languages:python dimstar_suse 2021-12-01 19:46:49 +00:00
  • 4e6d945d9a - update to 2.0.8: * Improvement over Vietnamese detection * MD improvement on trailing data and long foreign (non-pure latin) * Efficiency improvements in cd/alphabet_languages * call sum() without an intermediary list following PEP 289 recommendations * Code style as refactored by Sourcery-AI * Minor adjustment on the MD around european words * Remove and replace SRTs from assets / tests * Initialize the library logger with a NullHandler by default * Setting kwarg explain to True will add provisionally * Fix large (misleading) sequence giving UnicodeDecodeError * Avoid using too insignificant chunk * Add and expose function set_logging_handler to configure a specific StreamHandler dirkmueller 2021-11-29 11:18:31 +00:00
  • 905e9c7130 - update to 2.0.8: * Improvement over Vietnamese detection * MD improvement on trailing data and long foreign (non-pure latin) * Efficiency improvements in cd/alphabet_languages * call sum() without an intermediary list following PEP 289 recommendations * Code style as refactored by Sourcery-AI * Minor adjustment on the MD around european words * Remove and replace SRTs from assets / tests * Initialize the library logger with a NullHandler by default * Setting kwarg explain to True will add provisionally * Fix large (misleading) sequence giving UnicodeDecodeError * Avoid using too insignificant chunk * Add and expose function set_logging_handler to configure a specific StreamHandler dirkmueller 2021-11-29 11:18:31 +00:00
  • 380896adbc - require lower-case name instead of breaking build dirkmueller 2021-11-26 11:35:38 +00:00
  • 8ecb13326b - require lower-case name instead of breaking build dirkmueller 2021-11-26 11:35:38 +00:00
  • 515e72fd80 - Use lower-case name of prettytable package mcepl 2021-11-25 22:27:00 +00:00
  • e9dbb76923 - Use lower-case name of prettytable package mcepl 2021-11-25 22:27:00 +00:00
  • 80b5313625 Accepting request 927599 from devel:languages:python dimstar_suse 2021-10-27 20:21:05 +00:00
  • 08e2786e7f Accepting request 927599 from devel:languages:python dimstar_suse 2021-10-27 20:21:05 +00:00
  • fd5f5dc1f2 Accepting request 925848 from home:mnhauke dirkmueller 2021-10-26 20:41:42 +00:00
  • 6e97c41098 Accepting request 925848 from home:mnhauke dirkmueller 2021-10-26 20:41:42 +00:00
  • ef5560a5e2 Accepting request 894589 from devel:languages:python dimstar_suse 2021-05-20 17:25:29 +00:00
  • f48f1864e6 Accepting request 894589 from devel:languages:python dimstar_suse 2021-05-20 17:25:29 +00:00
  • d5d2d1f9e5 Accepting request 894588 from home:pgajdos:python mcalabkova 2021-05-20 09:54:40 +00:00
  • 0dfe8f434f Accepting request 894588 from home:pgajdos:python mcalabkova 2021-05-20 09:54:40 +00:00
  • 0ab2750707 Accepting request 870785 from devel:languages:python RBrownSUSE 2021-03-30 19:03:00 +00:00
  • 89ba0fb291 Accepting request 870785 from devel:languages:python RBrownSUSE 2021-03-30 19:03:00 +00:00
  • 4ec9d5c90b Accepting request 870710 from home:jayvdb:branches:devel:languages:python dirkmueller 2021-02-10 08:09:39 +00:00
  • e6af35c415 Accepting request 870710 from home:jayvdb:branches:devel:languages:python dirkmueller 2021-02-10 08:09:39 +00:00