auto_ef - auto encoding finder
/usr/bin/auto_ef [-e encoding_list] [-a] [-l level] [file ...]
The auto_ef utility identifies the encoding of a given file. The utility judges the encoding by using the iconv code conversion, determining whether a certain code conversion was successful with the file, and also by performing frequency analyses on the character sequences that appear in the file.
The auto_ef utility might produce unexpected output if the string is binary, a character table, a localized digit list, or a chronogram, or if the string or file is very small in size (for example, less than 100 bytes).
The auto_ef utility supports the below encodings:
Japanese PC Kanji, CP932, Shift JIS
Simplified Chinese EUC, GB2312
Simplified Chinese GB18030/GBK
Traditional Chinese EUC
Traditional Chinese. Extension of ISO-2022-CN
Hong Kong BIG5
West European, and similar
East European, and similar
Cyrillic, and similar
windows-1250, corresponding to ISO-8859-2
windows-1251, corresponding to ISO-8859-5
windows-1252, corresponding to ISO-8859-1
windows-1253, corresponding to ISO-8859-7
windows-1255, corresponding to ISO-8859-8
By default, auto_ef returns a single, most likely encoding for text in a specified file. To get all possible encodings for the file, use the –a option.
Also by default, auto_ef uses the fastest process to examine the file. For more accurate results, use the –l option.
To examine data with a limited set of encodings, use the –e option.
The following options are supported:
Shows all possible encodings in order of possibility, with scores in the range between 0.0 and 1.0. A higher score means a higher possibility. For example,
example% auto_ef -a test_file eucJP 0.89 zh_CN.euc 0.04 ko_KR.euc 0.01
Without this option, only one encoding with the highest score is shown.
Examines data only with specified encodings. For example, when encoding_list is specified as "ko_KR.euc:ko_KR.cp949", auto_ef examines text only with CP949 and ko_KR.euc. Without this option, auto_ef examines text with all encodings. Multiple encodings can be specified by separating the encodings using a colon (:).
Shows the usage message.
Specifies the level of judgment. The value of level can be 0, 1, 2, or 3. Level 3 produces the best result but can be slow. Level 0 is fastest but results can be less accurate than in higher levels. The default is level 0.
The following operands are supported:
File name to examine.
example% auto_ef file_nameExample 2 Examining encoding of a file at level 2.
example% auto_ef -l 2 file_nameExample 3 Examining encoding of a file with only eucJP or ko_KR.euc
example% auto_ef -e "eucJP:ko_KR.euc" file_name
The following exit values are returned:
An error occurred.
See attributes(7) for descriptions of the following attributes: