Character Encodings
Soli strings are UTF-8. The Encoding class converts between UTF-8 and legacy byte encodings (Latin-1 / ISO-8859-1 / Windows-1252, …) so you can import a non-UTF-8 file without turning accented characters into ?.
Overview
A byte like 0xE9 (é in Latin-1) is not valid UTF-8 on its own, so reading a Latin-1 file as text fails or garbles the accents. Decode the raw bytes from their actual charset first:
- Importing CSV / fixed-width exports from legacy systems
- Reading HTTP responses declared as
charset=ISO-8859-1 - Writing files back out in a charset a downstream system expects
Labels follow the WHATWG Encoding Standard — "latin1", "iso-8859-1", "windows-1252", "utf-8", … — and latin1/iso-8859-1 alias to windows-1252. An unknown label raises.
Static Methods
Encoding.decode(input, label)
Decodes input (a byte array Array<Int> or a string) from label into a UTF-8 string.
# café in Latin-1: c=99 a=97 f=102 é=233
Encoding.decode([99, 97, 102, 233], "latin1") # "café"
Encoding.decode([233], "iso-8859-1") # "é"Encoding.encode(string, label)
Encodes a UTF-8 string into a byte array (Array<Int>) in label. Characters not representable in the target charset become HTML numeric entities (e.g. an emoji → 😀).
Encoding.encode("café", "latin1") # [99, 97, 102, 233]
# round-trips
text = "Curaçao — déjà vu"
bytes = Encoding.encode(text, "windows-1252")
assert_eq(Encoding.decode(bytes, "windows-1252"), text)Common Use Cases
Importing a Latin-1 file
slurp and File.read accept a charset label directly, so importing is a one-liner.
# one step
text = slurp("clients.csv", "latin1")
text = File.read("clients.csv", "latin1")
# or explicitly via raw bytes
raw = slurp("clients.csv", "binary")
text = Encoding.decode(raw, "latin1")Exporting back to Latin-1
barf and File.write accept a byte array, so pair them with Encoding.encode.
barf("clients.csv", Encoding.encode(text, "latin1"))Error Handling
An unrecognized charset label raises — handle it the same way as any other error:
try
Encoding.decode(bytes, "no-such-encoding")
catch e
print("Error: " + e) # "Error: unknown encoding: no-such-encoding"
end