Go to main content

man pages section 1: User Commands

Exit Print View

Updated: Wednesday, February 10, 2021
 
 

auto_ef(1)

Name

auto_ef - auto encoding finder

Synopsis

/usr/bin/auto_ef [-e encoding_list] [-a] [-l level] 
     [file ...]
/usr/bin/auto_ef -h

Description

The auto_ef utility identifies the encoding of a given file. The utility judges the encoding by using the iconv code conversion, determining whether a certain code conversion was successful with the file, and also by performing frequency analyses on the character sequences that appear in the file.

The auto_ef utility might produce unexpected output if the string is binary, a character table, a localized digit list, or a chronogram, or if the string or file is very small in size (for example, less than 100 bytes).

The auto_ef utility supports the below encodings:

ASCII

ASCII

UTF-8

UTF-8

ISO-2022-JP

JIS

eucJP

Japanese EUC

PCK

Japanese PC Kanji, CP932, Shift JIS

ko_KR.euc

Korean EUC

ko_KR.cp949

Unified Hangul

ISO-2022-KR

ISO-2022 Korean

zh_CN.iso2022-CN

ISO-2022 CN/CN-EXT

zh_CN.euc

Simplified Chinese EUC, GB2312

zh_CN.gb18030

Simplified Chinese GB18030/GBK

zh_TW-big5

BIG5

zh_TW-euc

Traditional Chinese EUC

zh_TW.iso2022-CN-EXT

Traditional Chinese. Extension of ISO-2022-CN

zh_HK.hkscs

Hong Kong BIG5

iso-8859-1

West European, and similar

iso-8859-2

East European, and similar

iso-8859-5

Cyrillic, and similar

iso-8859-6

Arabic

iso-8859-7

Greek

iso-8859-8

Hebrew

CP1250

windows-1250, corresponding to ISO-8859-2

CP1251

windows-1251, corresponding to ISO-8859-5

CP1252

windows-1252, corresponding to ISO-8859-1

CP1253

windows-1253, corresponding to ISO-8859-7

CP1255

windows-1255, corresponding to ISO-8859-8

CP1256

windows-1256, Arabic

koi8-r

KOI8-R

TIS620.2533

TIS-620, Thai

CP874

CP874, Thai

By default, auto_ef returns a single, most likely encoding for text in a specified file. To get all possible encodings for the file, use the –a option.

Also by default, auto_ef uses the fastest process to examine the file. For more accurate results, use the –l option.

To examine data with a limited set of encodings, use the –e option.

Options

The following options are supported:

–a

Shows all possible encodings in order of possibility, with scores in the range between 0.0 and 1.0. A higher score means a higher possibility. For example,

example% auto_ef -a test_file
eucJP           0.89
zh_CN.euc       0.04
ko_KR.euc       0.01

Without this option, only one encoding with the highest score is shown.

–e encoding_list

Examines data only with specified encodings. For example, when encoding_list is specified as "ko_KR.euc:ko_KR.cp949", auto_ef examines text only with CP949 and ko_KR.euc. Without this option, auto_ef examines text with all encodings. Multiple encodings can be specified by separating the encodings using a colon (:).

–h

Shows the usage message.

–l level

Specifies the level of judgment. The value of level can be 0, 1, 2, or 3. Level 3 produces the best result but can be slow. Level 0 is fastest but results can be less accurate than in higher levels. The default is level 0.

Operands

The following operands are supported:

file

File name to examine.

Examples

Example 1 Examining encoding of a file
example% auto_ef file_name
Example 2 Examining encoding of a file at level 2.
example% auto_ef -l 2 file_name
Example 3 Examining encoding of a file with only eucJP or ko_KR.euc
example% auto_ef -e "eucJP:ko_KR.euc" file_name

Exit Status

The following exit values are returned:

0

Successful completion

1

An error occurred.

Attributes

See attributes(7) for descriptions of the following attributes:

ATTRIBUTE TYPE
ATTRIBUTE VALUE
Availability
text/auto_ef
Interface Stability
Committed

See Also

auto_ef(3EXT), libauto_ef(3LIB), attributes(7)