Skip to main content

Beware of Java's String.getBytes


Sometimes there are subtle bugs whose origin can be found in some quirks from the underlying language used to build the software. This blog post describes one of those cases in order to let both fellow security researchers and developers, who didn't know about it, become aware of this potential vulnerable pattern. In fact, I'm pretty sure that similar bugs to the one herein described likely affect a bunch of products/codebases out there.


In previous posts, I've already described some bugs in the Swiss Post's future E-voting system. While reading their Crypto-Primitives specification, which among other things describes the custom Hashing algorithm Swiss Post implemented, I noticed something potentially interesting.




Basically, there are 4 different types that are supported: byte arrays, strings, integers and vectors. Before being hashed, strings are converted to a byte array via the 'StringToByteArray' algorithm.


However, by comparing 'StringToByteArray' and 'ByteArrayToString' we can find a significant  difference: invalid UTF-8 sequences are only considered in the latter. Let's see how this was implemented in the code:

File: crypto-primitives-master/src/main/java/ch/post/it/evoting/cryptoprimitives/internal/utils/ConversionsInternal.java
079:    /**
080:     * See {@link ch.post.it.evoting.cryptoprimitives.utils.Conversions#stringToByteArray}
081:     */
082:    public static byte[] stringToByteArray(final String s) {
083:        checkNotNull(s);
084: 
085:        // Corresponds to UTF-8(S)
086:        return s.getBytes(StandardCharsets.UTF_8);
087:    }
088: 
089:    /**
090:     * See {@link ch.post.it.evoting.cryptoprimitives.utils.Conversions#byteArrayToString}
091:     */
092:    public static String byteArrayToString(final byte[] b) {
093:        checkNotNull(b);
094:        checkArgument(b.length > 0, "The length of the byte array must be strictly positive.");
095: 
096:        CharsetDecoder decoder = StandardCharsets.UTF_8.newDecoder();
097:        // The try-catch clause implements the pseudo-code's if statement
098:        try {
099:            // Corresponds to UTF-8^-1(B)
100:            return decoder.decode(ByteBuffer.wrap(b)).toString();
101:        } catch (CharacterCodingException ex) {
102:            throw new IllegalArgumentException("The byte array does not correspond to a valid sequence of UTF-8 encoding.");
103:        }
104:    }

As expected, at line 100, 'byteArrayToString' tries to decode the input to detect invalid UTF-8 sequences. On the other hand, at line 86, 'StringToByteArray' directly uses 'getBytes'.

Internally, 'getBytes' will encode the string before returning the bytes. Let's see Java's implementation:

File: java/lang/String.java
/**
     * Encodes this {@code String} into a sequence of bytes using the given
     * {@linkplain java.nio.charset.Charset charset}, storing the result into a
     * new byte array.
     *
     * <p> This method always replaces malformed-input and unmappable-character
     * sequences with this charset's default replacement byte array.  The
     * {@link java.nio.charset.CharsetEncoder} class should be used when more
     * control over the encoding process is required.
     *
     * @param  charset
     *         The {@linkplain java.nio.charset.Charset} to be used to encode
     *         the {@code String}
     *
     * @return  The resultant byte array
     *
     * @since  1.6
     */
    public byte[] getBytes(Charset charset) {
        if (charset == null) throw new NullPointerException();
        return encode(charset, coder(), value);
     }


As the description of the method clearly states, any invalid character sequence will be replaced (onMalformedInput(CodingErrorAction.REPLACE).onUnmappableCharacter(CodingErrorAction.REPLACE) with the default "replacement byte array",  it doesn't trigger any exception.

In Java, the replacement byte array for a default Charset provider is one of the Unicode specials, the replacement character (0xFFFD). However, for some reason Java's UTF-8 Charset Encoder uses '?' (63d).

File: 'sun.nio.cs.UTF_8'

This behavior does not seem to comply with the Unicode's security recommendations.


It also differs from other languages, such as C#, which follows the best-practice and replaces (Encoding.UTF8.GetBytes) the malformed character sequence with the UTF-8 encoded version (0xEF 0xBF 0xBD) of the replacement character.  

Impact

This specific pattern may result in different attack scenarios, depending on the logic where the vulnerable pattern has been identified.

Swiss Post E-Voting 

In this specific case, what we end up with is a hash collision vulnerability.
 
This silly  PoC elaborates how two different strings are not injectively encoded, thus leading to a potential hash collision according to the 'RecursiveHash' algorithm.


import java.nio.charset.StandardCharsets;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.util.Arrays;
import java.util.stream.Collectors;
import java.nio.charset.CharsetEncoder;
import java.nio.charset.CodingErrorAction;
import java.nio.ByteBuffer;
import java.nio.charset.CharacterCodingException;
import java.nio.CharBuffer;

public class poc {

    public static void encode(String string)
     throws CharacterCodingException {
     CharsetEncoder encoder = StandardCharsets.UTF_8.newEncoder();

     System.out.println("\nEncoding");

     ByteBuffer encoded = encoder.encode(CharBuffer.wrap(string.toCharArray()));

     System.out.println("\nEncoding end");

    }

    public  static void printhex(byte []ars) {

       for(int i=0;i<ars.length;i++){  
            System.out.print(String.format("\\x%04X",(short)ars[i]));  
        }

        System.out.print("\n");
        return;
    }

    public static byte[] stringToByteArray(final String s) {

        return s.getBytes(StandardCharsets.UTF_8);
    }

    public static String  buildCustomString(String dat) {

        String badString = Arrays.stream(dat.split("\\+U"))
                .filter(s -> ! s.isEmpty()) 
                .map(s -> {
                    try {
                        return Integer.parseInt(s, 16);
                    } catch (NumberFormatException e) { 
                        System.out.println("Error parsing int");
                    }
                    return null; 
                })
                .map(i -> Character.toString(i)) 
                .collect(Collectors.joining());

        return badString;
    }

    public static String DumpInfo(String inf)
    {
        String badString;

        System.out.println("=>> Original string (custom format): \"" + inf+"\""); 
        System.out.println("[+] Building String");
        badString = buildCustomString(inf);

        System.out.println("[+] String badString => \""+ badString+"\"");
        System.out.print("[+] badString.toCharArray() =>\t\t"); 
        char[] hash3 = badString.toCharArray();

        for(int i=0;i<hash3.length;i++){  
            System.out.print(String.format("\\x%04X",(short)hash3[i]));  
        }  

            System.out.print("\n");

        System.out.print("[+] stringToByteArray(badString) =>     "); 
        printhex(stringToByteArray(badString));

        return badString;
    }

    public static void main(String args[]) {

        String badString1,badString2;

        String base = "Question_1?";

        // +UD8AF is illegal so it will be mapped to '?' after UTF8 encoding.
        String data1 = "+U0051+U0075+U0065+U0073+U0074+U0069+U006f+U006E+U005f+U0031+UD8AF";

        badString1 = DumpInfo(data1);

        System.out.println("[+] badString after UTF-8 encoding: \""+ badString1 + "\"");
        System.out.print("[+] stringToByteArray(base) =>   \t"); 
        printhex(stringToByteArray(base));

        if(!badString1.equals(base)){
            System.out.println("[+] 'badString' and 'base' are not equal");
        }

        try {
            //This will trigger an exception
            encode(badString1);

        }
        catch (Exception e) {
            e.printStackTrace();
        }

        System.out.print("\n");

    }

}

In the PoC there are two strings 'base' and 'badString'. The first one is a valid UTF-8 string that contains "Question_1?", while the latter contains "Question_1" plus a malformed UTF-8 sequence ('\uD8AF').

When 'stringToByteArray' is invoked on 'badString', its illegal sequence will be substituted by '?' resulting in the same hash that would be generated for the 'base' string. However, those strings, before being encoded, are different, as can be checked in the 'toCharArray()' and 'equals' comparisons. This is important to be able to potentially bypass certain checks, as 'equals' operates at the char array level

The output of the PoC is as follows

Swiss Post confirmed the bug ( #YWH-PGM232-122 ), which has been patched in version 1.3.0

Other products

The fact that Java's UTF-8 Encoder is replacing a malformed character sequence with a valid one, '?', which in turn plays an important role in URLs  makes me think that probably, in addition to the potential cryptographic issues we have already seen, this vulnerable pattern might be used to bypass certain security-related logic.

As long as the attacker can control the string, for instance after deserializing an attacker controlled JSON by using Jackson, there will be a chance to abuse the Java's UTF-8 'getBytes' replacement logic.

 ObjectMapper objectMapper = new ObjectMapper();
 String json = "{ \"badString\" : \"Question_1\\uD84F\" }";
 JsonNode jsonNode = objectMapper.readTree(json);
 String badString = objectMapper.readValue(jsonNode.get("badString").toString(), String.class);

If you happen to stumble upon a product with such a vulnerability, I would love to hear about it.





Popular posts from this blog

What Really Happened in Chernobyl During the Beginning of the Russian Invasion?

This blog post contains the web version of my research paper: " Seeing Through the Invisible: Radiation Spikes Detected in Chernobyl During the Russian Invasion Show Possible Evidence of Fabrication ", which was unveiled at BlackHat USA 2023 . It is intended to ease the indexing and dissemination of the information collected during this research.  In a few days, I'll be in Brussels presenting this research.  The original paper (PDF) can be downloaded here . Additional references: https://www.wired.com/story/chernobyl-radiation-spike-mystery/  (Kim Zetter) https://www.zetter-zeroday.com/p/radiation-spikes-at-chernobyl-a-mystery  (Kim Zetter) https://medium.com/war-notes/chornobyl-3-92216d21b223  (Olegh Bondarenko) INDEX Foreword Executive summary Introduction 1. Physical      1986      Resuspension      Transport      Humidity      Traffic 2. Cyber    ...
Read more

De-Anonymization attacks against Proton services

  In November 2021 YesWeHack invited me to participate in a private bug bounty program organized by  Bug Bounty Switzerland on behalf of Proton AG.  The scope of the program was quite interesting and heterogeneous, as it covered most of the applications and services offered by Proton, such as ProtonMail and ProtonVPN. As a result, multiple technologies and codebases were in scope, ranging from typescript, in the open-source part of Protonmail, to .NET/Swift used by ProtonVPN apps for Windows and macOS respectively. Proton is well-known for its privacy-driven services offer, so they are based on Switzerland where the legislation seems to match Proton's requirements to provide that kind of services: thus maximizing the privacy of their communications, minimizing the amount of data they log from their users while keeping a law-abiding status.  It wouldn't be realistic to think of Proton users as an homogenous group; you may be using Proton because you're genuinely w...
Read more

Finding vulnerabilities in Swiss Post's e-voting system: part 3

Exactly two years ago I brought my blog back to life, after many years of hiatus, with " Finding vulnerabilities in Swiss Post’s future e-voting system - Part 1 ". That was the first of a series of blog posts covering that system. During these two years I've been periodically assessing the security posture of this e-voting solution, as part of their Bug Bounty program , which I personally recommend.   Since the first time I reviewed their codebase a lot of things have changed, for good, as many areas have been dramatically improved. To be honest, from a security perspective the codebase back then was kind of a mess.   When the first Swiss Post e-voting platform was published, back in 2019, it faced some public scrutiny, mostly from the academic community.  As a result, some significant issues were uncovered , so eventually Swiss Post decided to suspend the deployment of the system. That first version had been developed by Scytl , Spanish company specializ...
Read more

Losing control over Schneider's EcoStruxure Control Expert

  During Q2 2022, in view of the geopolitical situation that unfolded after the Russian invasion of Ukraine, I decided that it wouldn't do any harm to kill some bugs in some of the main players within the ICS arena. I focused in those software frameworks that are running on the engineering workstations so, if compromised, attackers would be in a privileged position to manipulate controllers logic, thus enabling sophisticated attacks with a potential physical impact (i.e triton). I responsibly reported a bunch a unauthenticated remotely exploitable bugs to the corresponding vendors. In one case, after being ignored for months, I had to resort to the 'twitter, do your magic' approach and tweeted that I would be disclosing the issues if the situation persisted. It took just few hours for the vendor to get back to me. The positive side is that they found the bugs interesting and all that mess ended up in paid work.   This blog post covers a similar scenario in a different ven...
Read more