I was having some very weird errors when I was testing out TT-RSS API. One of the methods just would not work and server reported HTTP/500 error. I figured out that the error is only there, if I use non-ASCII characters in my POST data.

My curl one-line to call the API was something like:

curl -H "Content-Type: application/json; charset=utf-8" -d '{"op":"shareToPublished","title":"ččč","url":"https://aaa.cz","content":"aaa","sid":"XXX"}' 'https://TT_RSS_SERVER/api/'

I was thinking that, OK, I have charset=utf-8 in my Content-Type and that’s it, curl should encode input data correctly, right? At least, that was what some clever people suggested on StackOverflow.

Well, then I decided to see what is actually sent with --trace option and that clearly showed, that č characters are encoded as e8 (HEX) which is not what I would expect.

I subsequently checked my command line “encoding” with locale command and that was OK, I had cs_CZ.UTF-8 there. It seems, that for some reason, curl was encoding my data as windows-1250, not UTF-8.

Therefore I tried to do the encoding stuff by myself with great iconv tool, resulting in this:

curl -H "Content-Type: application/json; charset=utf-8" --data "$(echo -n '{"op":"shareToPublished","title":"ččč","url":"https://aaa.cz","content":"aaa","sid":"XXX"}' | iconv -f windows-1250 -t utf-8)" 'https://TT_RSS_SERVER/api/'

And, this worked! So, at least I finally learned how to actually send real UTF-8 POST requests.