Java unicode conversion

Question

0.00/5 (No votes)

See more:

I am having a problem for the Unicode conversion. The main problem was the String parameter passed was something unsupported characters.

Here is the sample of the code:

I have a batch script which will be executed on a server.
By the meantime, execution status log should be provided. This script contains some Japanese character as mentioned below which actually need to be displayed as it is in the log. For which I am having a problem.

In this, the topmost string "String command = sshcommand.getCommand();" containssome "Remote command with parameters as string".

What I have tried:

package jp.co.kentaku.kikan.util.ssh;

import static java.nio.charset.StandardCharsets.UTF_8;

import java.io.InputStream;
import java.nio.charset.Charset;
import java.util.Map;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.csc.qre.iseries.cl.CallCommand.ParamKey;
import com.jcraft.jsch.Channel;
import com.jcraft.jsch.ChannelExec;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;


/**
 * SSH Handler for ssh communication with servers Linux as well as Windows
 * @author ssharma365
 *
 */
public class SSHHandler {
	/**
	 * Enabling logger for this class
	 * 
	 */
	private static final Logger logger = LoggerFactory.getLogger(SSHHandler.class);

	/**
	 * Jsch channel
	 */
	private JSch jschSSHChannel;
	/**
	 * Remote machine User name
	 */
	private String strUserName;
	/**
	 * Remote machine server name or IP
	 */
	private String strConnectionIP;
	/**
	 * The port on which the remote machine will open the ssh communiction. by default 22
	 */
	private int intConnectionPort;
	/**
	 * Remote password for the above user
	 */
	private String strPassword;
	/*
	 * SSH session reference
	 */
	private Session sesConnection;
	/**
	 *  Time out 
	 */
	private int intTimeOut;

	private void doCommonConstructorActions(String userName, String password, String connectionIP,
			String knownHostsFileName) {
		jschSSHChannel = new JSch();

		try {
			jschSSHChannel.setKnownHosts(knownHostsFileName);
		} catch (JSchException jschX) {
			logError(jschX.getMessage());
		}

		strUserName = userName;
		strPassword = password;
		strConnectionIP = connectionIP;
	}

	public SSHHandler(String userName, String password, String connectionIP, String knownHostsFileName) {
		doCommonConstructorActions(userName, password, connectionIP, knownHostsFileName);
		intConnectionPort = 22;
		intTimeOut = 60000;
	}

	
	public String connect() throws Exception {
		String errorMessage = null;

		
			sesConnection = jschSSHChannel.getSession(strUserName, strConnectionIP, intConnectionPort);
			sesConnection.setPassword(strPassword);
	
			sesConnection.setConfig("StrictHostKeyChecking", "no");
			sesConnection.connect(intTimeOut);
		
		return errorMessage;
	}

	private String logError(String errorMessage) {
		if (errorMessage != null) {
			logger.error(strConnectionIP + ":" + intConnectionPort + " - " + errorMessage);
			}
		return errorMessage;
	}

	private String logWarning(String warnMessage) {
		if (warnMessage != null) {
			logger.error(strConnectionIP + ":" + intConnectionPort + " - " + warnMessage);
			}
		return warnMessage;
	}

	public String sendCommand(String command) throws Exception {
		String errorMsg = null;
		StringBuilder outputBuffer = new StringBuilder();

		
			Channel channel = sesConnection.openChannel("exec");
			((ChannelExec) channel).setCommand(command);
			InputStream commandOutput = channel.getInputStream();
			InputStream commandErrorOutput = ((ChannelExec) channel).getErrStream();
			channel.connect();
			int readByte = commandOutput.read();

			while (readByte != 0xffffffff) {
				outputBuffer.append((char) readByte);
				readByte = commandOutput.read();
			}

			int readErrorByte = commandErrorOutput.read();
			boolean error = false;
			while (readErrorByte != 0xffffffff) {
				error = true;
				outputBuffer.append((char) readErrorByte);
				readErrorByte = commandErrorOutput.read();
			}
			channel.disconnect();
			int exitStatus = channel.getExitStatus();
			if (error) {
				errorMsg = outputBuffer.toString();
				logger.error("Exception occured while executing the commnad:(" + command + ") exitStatus received:"
						+ exitStatus + " Message:" + errorMsg);
				logger.error("Message Id:CPF91CB Message:" + errorMsg);
				//throw new Cpf91cb("CPF91CB", errorMsg);
				throw new Exception(errorMsg);
			}
		

		return outputBuffer.toString();
	}

	public void close() {
		sesConnection.disconnect();
	}

	public static void executeSSHCommand(SSHCommand sshcommand) throws Exception {
		logger.info("Entering the executeSSHCommand()");
		/**
		 * YOU MUST CHANGE THE FOLLOWING FILE_NAME: A FILE IN THE DIRECTORY USER: LOGIN
		 * USER NAME PASSWORD: PASSWORD FOR THAT USER HOST: IP ADDRESS OF THE SSH SERVER
		 **/
		String command = sshcommand.getCommand();
												
		String userName = sshcommand.getUserName();
		String password = sshcommand.getPassword();
		String connectionIP = sshcommand.getServerName();
		String ccsid = sshcommand.getCcsid();
		logger.info("Attempting connection to Server:" + connectionIP + " user:" + userName + " using password:"
				+ password);
		
		SSHHandler instance = new SSHHandler(userName, password, connectionIP, "");
		String errorMessage = "";
		try {
			errorMessage = instance.connect();
			logger.info("Established connection to Server:" + connectionIP + " user:" + userName + " using password:"
					+ password);
		} catch (Exception e) {
			logger.error("Message ID : CPF91CB ; Message: Could not connect to Server:" + connectionIP + " user:"
					+ userName + " using password:" + password);
			//throw new Cpf91cb("CPF91CB", "Could not connect to the server :" + connectionIP + " with User:" + userName,e);
			throw e;
		}
		if (errorMessage != null) {
			logger.error("Could not connect to the server :" + connectionIP + " with User:" + userName + " message:"
					+ errorMessage);
			/*throw new Cpf91cb("CPF91CB", "Could not connect to the server :" + connectionIP + " with User:"
					+ userName + " message:" + errorMessage, new Exception(errorMessage));*/
			throw new Exception(errorMessage);
		}

		// call sendCommand for each command and the output
		// (without prompts) is returned
		String result = null;
		try {
			logger.info("Trying to execute command" + command + " on " + connectionIP);
			result = instance.sendCommand(command);
			logger.info("Executed command :" + command + " on Server:" + connectionIP + " user:" + userName
					+ " using password:" + password);
			logger.info("CHARSET found=" + ccsid);
			if (ccsid != null && ccsid.equals("943")) {
				logger.info("Lined up for conversion");
				result = convertUTF8ToShiftJ(result);
			}
			logger.info("Command output : " + result);
		} catch (Exception e) {
			errorMessage = "Exception while executing the command:" + command + " on Server:" + connectionIP;
			errorMessage = errorMessage + " message:" + e.getMessage();
			// e.printStackTrace();
			logger.error(errorMessage);
			/*throw new Cpf91cb("CPF91CB", "Could not connect to the server :" + connectionIP + " with User:"
					+ userName + " message:" + errorMessage,new Exception(errorMessage));*/
			throw e;
		}
		// close only after all commands are sent
		instance.close();
	}

	private static String convertUTF8ToShiftJ(String uft8Str) {
		String shftJStr = null;
		try {
			
			byte[] b = uft8Str.getBytes(UTF_8);
			shftJStr = new String(b, Charset.forName("SHIFT-JIS"));
			logger.info("Converted to the string :" + shftJStr);
			System.out.println(shftJStr);
		} catch (Exception e) {
			e.printStackTrace();
			return uft8Str;
		}
		return shftJStr;
	}

	/**
	 * Get parameter value.
	 * 
	 * @param params
	 *            set of parameters
	 * @param paramKey
	 *            parameter key to retrieve value
	 * @param defaultValue
	 *            default value if value is null or not found
	 * @return
	 */
	@SuppressWarnings("unchecked")
	public static <T> T getParam(final Map<ParamKey, Object> params, final ParamKey paramKey, final T defaultValue) {
		if (params == null) {
			return defaultValue;
		}

		T value = (T) params.get(paramKey);
		if (value == null) {
			return defaultValue;
		}

		return value;
	}

	/**
	 * Get parameter value.
	 * 
	 * @param params
	 *            set of parameters
	 * @param paramKey
	 *            parameter key to retrieve value
	 * @param defaultValue
	 *            default value if value is null or not found
	 * @return
	 */
	@SuppressWarnings("unchecked")
	public static <T> T getParam(final Map<ParamKey, Object> params, final ParamKey paramKey) {
		if (params == null) {
			return null;
		}

		return (T) params.get(paramKey);
	}

}

Output:

*** UX0025.SH ﾂ開ﾂ始ﾂ　ﾂ（startedﾂ）
*** UX0025.SH ﾂ偲?ﾂ行ﾂ陳?ﾂ（executing...ﾂ）
*** UX0025.SH ﾂ終ﾂ猟ｹﾂ　ﾂ（endedﾂ　ﾂ）

But Actually it supposed to be:

*** UX0025.SH 開始　（started）
*** UX0025.SH 実行中（executing...）
*** UX0025.SH 終了　（ended　）

The problem seems due to the string parameter "uft8Str" which carries some unsupportive characters. So someone please help me out with this.
Thanks.

Posted 22-Aug-18 17:22pm

Member 13958412

Updated 24-Aug-18 0:59am

v6

Add a Solution

Comments

Richard MacCutchan 23-Aug-18 3:55am

Your code does not match the output that you show, and gives no information about the input.

Member 13958412 23-Aug-18 5:56am

Hi, I have tried to modified my question. Hope it will make a bit clear.

Jochen Arndt 24-Aug-18 5:48am

I don't get it why you want to convert a Unicode string to a multi byte string using a specific code page. This should be only necessary if you have to pass it with that specific code page. But all you are doing is passing it to a logger. That will usually convert the input to the encoding used internally by the logger which is very probable Unicode.

A different scenario would be having a multi byte string with a specific encoding as input. Then convert that to Unicode for further processing.

If you are converting a Unicode string to a non Unicode code page / CCSID, all characters not supported by that code page are lost.

Member 13958412 24-Aug-18 6:15am

OK, as you mentioned I am not passing through it, instead, if I do [String shftJStr = new String(uft8Str.getBytes());] then, I will be getting the result as:
*** UX0025.SH �J�n�@�istarted�j
*** UX0025.SH �À�s��iexecuting...�j
*** UX0025.SH �I�¹�@�iended�@�j

Jochen Arndt 24-Aug-18 6:27am

And what happens if you log the result as it is without applying any conversion?

Member 13958412 24-Aug-18 6:35am

Without conversion also seems similar output as above.

Jochen Arndt 24-Aug-18 6:42am

The problem is sourced in your sendCommand function() because that does not return a valid (UTF-16 encoded) Java string.

I will write an answer.

Member 13958412 24-Aug-18 6:43am

Thank you. It will be helpful.

Member 13958412 24-Aug-18 19:59pm

Hi, sorry now I am out of server access zone. So I would be able to implement it only on Monday. Then I will come back to you.

Member 13958412 27-Aug-18 2:39am

Hi, I tried to implement the logic of byte buffer.
Instead of "result = instance.sendCommand(command);", if I implement "String result = new String(buffer, 0, rxCount, "UTF8");", there will be remote execution error. May be implementation was wrong or what!!

Jochen Arndt 27-Aug-18 2:50am

You have to implement it in your sendCommand() function using the byte buffer instead of the StringBuilder.

Member 13958412 27-Aug-18 23:23pm

Hi! Sorry to say that I tried all the way but it didn't really work for me. First of all utf8 was not supported so I turned it to 16. But still the output is not displaying the correct one.

Jochen Arndt 28-Aug-18 2:56am

At first you have to know which encoding is used by the service (send via network) but UTF-8 is common. Then you have to convert that to a Java String which is UTF-16 internally. UTF-8 is supported by Java. If it does not work, it might be also sourced in your receiving code.

Again:
What you are receiving are bytes. Always. The meaning of the bytes is defined by the used protocol. With SSH it is text which is usually UTF-8 encoded. If you need to process that with Java as string, you have to convert it to a Java String as shown in my solution using the encoding used by the server.

Member 13958412 28-Aug-18 23:22pm

Thanks for the guidance sir!

1 solution

Add a Solution

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Jochen Arndt · Accepted Answer · 2018-08-24T01:00:00

The problem is sourced in your sendCommand() function because that does not return a valid Java String (UTF-16 encoded).

The received data are bytes. So you have to use a byte buffer for receiving.

Untested example from scratch:

Java

// Assuming knowing the max. response length for simplicity
byte[] buffer = new byte[maxResponseLength];
int rxCount = 0;

// Receive data into buffer using and incrementing rxCount accordingly

// Create a Java String from the data.
// This requires knowing the encoding of the data which is probably UTF-8 
//  for responses from SSH servers.
String result = new String(buffer, 0, rxCount, "UTF8");

Java unicode conversion

1 solution

Solution 2

Add your solution here

Preview 0

Java unicode conversion

1 solution

Solution 2

Add your solution here

Preview 0

Existing Members

...or Join us