Question

Good day.

The one thing I now hate about Haskell is quantity of packages for working with string.

First I used native Haskell [Char] strings, but when I tried to start using hackage libraries then completely lost in endless conversions. Every package seem to use different strings implementation, some adopts their own handmade thing.

Next I rewrote my code with Data.Text strings and OverloadedStrings extension, I chose Text because it has a wider set of functions, but it seems many projects prefer ByteString.
Someone could give short reasoning why to use one or other?

PS: btw how to convert from Text to ByteString?

Couldn’t match expected type
Data.ByteString.Lazy.Internal.ByteString
against inferred type Text
Expected type: IO Data.ByteString.Lazy.Internal.ByteString
Inferred type: IO Text

I tried encodeUtf8 from Data.Text.Encoding, but no luck:

Couldn’t match expected type
Data.ByteString.Lazy.Internal.ByteString
against inferred type Data.ByteString.Internal.ByteString

UPD:

Thanks for responses, that *Chunks goodness looks like way to go, but I somewhat shocked with result, my original function looked like this:

htmlToItems :: Text -> [Item]
htmlToItems =
    getItems . parseTags . convertFuzzy Discard "CP1251" "UTF8"

And now became:

htmlToItems :: Text -> [Item]
htmlToItems =
    getItems . parseTags . fromLazyBS . convertFuzzy Discard "CP1251" "UTF8" . toLazyBS
    where
      toLazyBS t = fromChunks [encodeUtf8 t]
      fromLazyBS t = decodeUtf8 $ intercalate "" $ toChunks t

And yes, this function is not working because its wrong, if we supply Text to it, then we’re confident this text is properly encoded and ready to use and converting it is stupid thing to do, but such a verbose conversion still has to take place somewhere outside htmltoItems.

Answer 1

ByteStrings are mainly useful for binary data, but they are also an efficient way to process text if all you need is the ASCII character set. If you need to handle unicode strings, you need to use Text. However, I must emphasize that neither is a replacement for the other, and they are generally used for different things: while Text represents pure unicode, you still need to encode to and from a binary ByteString representation whenever you e.g. transport text via a socket or a file.

Here is a good article about the basics of unicode, which does a decent job of explaining the relation of unicode code-points (Text) and the encoded binary bytes (ByteString): The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets

You can use the Data.Text.Encoding module to convert between the two datatypes, or Data.Text.Lazy.Encoding if you are using the lazy variants (as you seem to be doing based on your error messages).