How to address issues with 'unmappable characters'?

Article ID:360004397911
2 minute readKnowledge base

Issue

  • We see a stacktrace that shows a cause like the one shown below:

java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: <pathOfFileWithUnmappableCharacters>
  • How to enable unicode support in Jenkins build logs

  • How to enable UTF-8 in Jenkins build logs

Resolution

The root cause of this issue is most likely due to the encoding used by the JVM running the process.

If the exception appears in the Jenkins logs, the Jenkins controller’s JVM is most likely the culprit. If the exception appears in a build console output, a build agent’s JVM is most likely the culprit.

Check the encoding of the JVM

Check the encoding of the JVM by looking at the following system properties:

  • file.encoding

  • sun.jnu.encoding

The system properties can be found in Jenkins:

  • for the Jenkins controller, under Manage Jenkins  System Information

  • for a Jenkins agent, under Manage Jenkins  Manage Nodes  NODE_NAME  System Information

If the encoding is not UTF-8, this is most likely the root cause of the problem.

Solution

Jenkins controller

To fix this, add the system properties -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 and restart the process:

  • for the Jenkins controller, add the properties as explained in How to add Java arguments to Jenkins? and restart Jenkins

  • for a Jenkins agent, add the properties to the "JVM options" of the agent’s configuration and restart the agent

Agent

Open the agent’s configuration page in Jenkins UI, and click the Advanced…​ options. Set the JVM options as follows and restart the agent

unicode linux jnlp

Check the Locale of the machine running the JVM

If the above solution did not work, it could be that the encoding of the JVM is enforced by the Locale of the underlying OS. If that’s the case, setting the encoding via system properties would have no effect.

Check the Locale of the system by looking at the environment variables. Open a linux terminal and run

env | grep LANG

Depending on your geographical location, the output would be similar to

LANG=en_US.UTF-8
GDM_LANG=en_US
LANGUAGE=en_US:en

If the encoding is not <country>.UTF-8 for LANG, it needs to be changed.

en_US is for US English language but the language may vary.

Solution

To fix this, change the Locale of the machine and restart the process. Depending on the distribution and the environment, the solution differs. Here are a couple of relevant articles:

Changes made to the locale are applied when a user logs in. You will need to logout and login again for the changes to take effect. Then restart the JVM process.