Format of export data (ISO-8859-1 or UTF-8)

Occasional Contributor

Format of export data (ISO-8859-1 or UTF-8)

Hi Product team,

 

What is the format of data when exported as .CSV or .TXT?  Is it ISO-8859-1 or UTF-8.  What I have seen is if the module is English only it outputs as ISO-8859-1 and if the module is Chinese, output is UTF-8.  Can we have some clarity on this please?  Thanks in advance.

 

-Regimon Sebastian

10 REPLIES 10
Frequent Contributor

Re: Format of export data (ISO-8859-1 or UTF-8)

@sebastian,

 

This article in Anapedia explains which special characters you can import and those you should avoid. It also provides instructions on how to encode import files containing special characters.

 

https://help.anaplan.com/anapedia/Content/Import_and_Export/Get_Started_with_Imports/Tips_for_Import...

 

Thanks

Sathya

Message 2 of 11
Occasional Contributor

Re: Format of export data (ISO-8859-1 or UTF-8)

Hi @Sathya ,

I am not referring to import data. I am referring to export files.

Is the exported data in ISO-8859-1 or UTF-8 format?

 

Regimon

Message 3 of 11
Highlighted
Expert

Re: Format of export data (ISO-8859-1 or UTF-8)

It is always in UTF-8. If a program reads it as a file it may try to autodetect the encoding, and if it only has ASCII characters it may think ISO-8859-1 is a good match.

Message 4 of 11
Occasional Contributor

Re: Format of export data (ISO-8859-1 or UTF-8)

That is what I thought too .. but when I imported back the same file I exported out (export format expected to be UTF-8), Anaplan auto detected it as ISO-8859-1 instead of UTF-8. Is it import or export that is getting it wrong?

Message 5 of 11
Community Boss

Re: Format of export data (ISO-8859-1 or UTF-8)

Hi Sebastian, 

 

The encoding character set is used to make possible to save special language characters in TXT files (extensions of files CSV TXT, etc.).

For instance, if the file is setup as "ISO-8859-1" you will not able to see correctly the Russian alphabet but if the file is saved as "UTF-8" the special characters are correct viewed. 

 

Another particularity: "UTF-8" character set is including the "ISO-8859-1". "UTF-8" is just an extension of "ISO-8859-1".

 

So, simplifying: you can always setup the Anaplan import action with "UTF-8". If the file that you import it is saved with "ISO-8859-1" the file will still be able to be read correctly and loaded in Anaplan without an issue.

 

When creating an Anaplan import action if the file is recognized as "ISO-8859-1" I would always change it manually to "UTF-8" for the simple reason that the "UTF-8" includes the "ISO-8859-1". 

If the Anaplan action is saved using "ISO-8859-1" and you try to load a "UTF-8" file - the import process will not work. 

 

UTF means "Unicode Transformation Format". 

 

To have a file in UTF-8 from Excel use the option "CSV UTF-8 (Comma delimited) (*.csv) option when saving a file. 

 

However if from Excel, the file is saved using the option "Unicode Text(*.txt)" the encoding of the saved txt file will UTF-16LE.  Even is still a UTF type encoding, the "UTF-16LE" and "UTF-8" encoding are not compatible between them.   

 

Ciao

Alex

 

Message 6 of 11
Occasional Contributor

Re: Format of export data (ISO-8859-1 or UTF-8)

Hi Alex,

As I had stated earlier, I am not referring to import data. I am referring to export file format.

 

Regi

Message 7 of 11
Community Boss

Re: Format of export data (ISO-8859-1 or UTF-8)

Sorry, 

 

You are correct. As @ben_speight  mentioned the Anaplan export is always in UTF-8. 

 

I made also some tests with simple data and both CSV and TXT exported files were always in UTF-8, even in the module were present only English characters. 

How did you verify the encoding of the exported file? 

Message 8 of 11
Occasional Contributor

Re: Format of export data (ISO-8859-1 or UTF-8)

Hi

I imported back and Anaplan detected some files as ISO-8859-1 and some as UTF-8.

 

rgds

Regi

Message 9 of 11
Community Boss

Re: Format of export data (ISO-8859-1 or UTF-8)

Hi Regi, 

 

You are correct again. Interesting behavior indeed. 

 

But, if you check the exported form Anaplan file using Notepad++ in menu "Encoding" you will notice that it is checked "Encode in UTF-8" also for only English character files. 

 

I always change an import action in Anaplan from encoding "ISO-8559-1" or any Windows encoding that is automatically detected by Anaplan into "UTF-8". 

 

"ISO-8559-1" and other Windows encoding are included in "UTF-8".  

 

However, if the file is encoded "UTF-16LE" then it is needed to be used in Anaplan action "UTF-16LE" as well. 

 

Regards, 

Alex

 

 

Message 10 of 11