注：本文使用的都是UTF-8的编码encoderjava的String字符转byte[]
byte[] abyte0 = new byte[4]; Listutf8mb4String = new ArrayList<>(); for (char i = Character.MIN_HIGH_SURROGATE; i <= Character.MAX_HIGH_SURROGATE; i++) { for (char j = Character.MIN_LOW_SURROGATE; j <= Character.MAX_LOW_SURROGATE; j++) { int character = Character.toCodePoint(i, j); abyte0 = new byte[4]; abyte0[0] = (byte) (240 | character >> 18); abyte0[1] = (byte) (128 | character >> 12 & 63); abyte0[2] = (byte) (128 | character >> 6 & 63); abyte0[3] = (byte) (128 | character & 63); String str = new String(abyte0); utf8mb4String.add(str); System.out.println(str); } }
4个字节的"字符"在这里是全部都包含了，你可以复制粘贴到你的编辑器里面执行，取一个4字节"字符",用单引号引着，编译会出现错误，因为4个字节的"字符"并不是一个char组成的，而是俩个，想了解下面有详述。
/** The value is used for character storage. */ private final char value[];
字符串内部存储其实是一个char[]总结一下以下的分析：(#：代表0或者1) 1）单字节字符 字符c属于范围[0x0000 , 0x00C8) byte[]规则： b[0] = #### ####(但不大于0xC8) 2）双字节字符 字符c属于范围[0x00C8 , 0x0800)
byte[]规则：
高6位 : b[0] = 110# #### 低6位：b[1] = 10## #### 3）3个字节的字符 字符c属于范围其他情况的补集
byte[]规则：
高4位 : b[0] = 1110 #### 中6位：b[1] = 10## #### 低6位：b[2] = 10## #### 4）4个字节的"字符" 字符c属于范围[0xD800 , 0xDFFF](高位字符范围为：[0xD800,0xDBFF]，低位字符范围为：[0xDC00,0xDFFF])
byte[]规则：
高8位 : b[0] = 1111 0### 中高6位：b[1] = 10## #### 低高6位：b[2] = 10## #### 低6位: b[3] = 10## ####注：java是大端，低地址存储高位，高地址存储低位。 (b[i] & 0xF8)== 0xF0可以判断这个字符串是否是包含4个字节的"字符"。I.encoder constructor
public byte[] getBytes(String charsetName) throws UnsupportedEncodingException { if (charsetName == null) throw new NullPointerException(); return StringCoding.encode(charsetName, value, 0, value.length); } 以上是String.getBytes方法代码，用于转化字符为byte[]的。 private StringEncoder(Charset cs, String rcn) { this.requestedCharsetName = rcn; this.cs = cs; this.ce = cs.newEncoder() .onMalformedInput(CodingErrorAction.REPLACE) .onUnmappableCharacter(CodingErrorAction.REPLACE); this.isTrusted = (cs.getClass().getClassLoader0() == null); }
以上是StringEncoder的构造函数，注意到
.onMalformedInput(CodingErrorAction.REPLACE) .onUnmappableCharacter(CodingErrorAction.REPLACE);
StringEncoder默认是
CodingErrorAction.REPLACE
作用是，当发生无法满足规则去转化byte[]的时候，就需要使用代替字符来代替无法转化的字符。
this.ce = cs.newEncoder()
protected CharsetEncoder(Charset cs, float averageBytesPerChar, float maxBytesPerChar) { this(cs, averageBytesPerChar, maxBytesPerChar, new byte[] { (byte)'?' }); } 以上是CharsetEncoder的改造函数，这是UTF_Encoder构造所调用到的构造函数，可以看到replacement字符是'?'字符。默认的。II.encode chars to byte[]
public int encode(char ac[], int i, int j, byte abyte0[]) { int k = i + j; int l = 0; for(int i1 = l + Math.min(j, abyte0.length); l < i1 && ac[i] < '\200';) abyte0[l++] = (byte)ac[i++]; while(i < k) { char c = ac[i++]; if(c < '\200')//1 abyte0[l++] = (byte)c; else if(c < '\u0800')//2 { abyte0[l++] = (byte)(192 | c >> 6); abyte0[l++] = (byte)(128 | c & 63); } else if(Character.isSurrogate(c))//4 { if(sgp == null) sgp = new Surrogate.Parser(); int j1 = sgp.parse(c, ac, i - 1, k);//4.1 if(j1 < 0)//4.2 { if(malformedInputAction() != CodingErrorAction.REPLACE) return -1; abyte0[l++] = replacement()[0]; } else//4.3 { abyte0[l++] = (byte)(240 | j1 >> 18); abyte0[l++] = (byte)(128 | j1 >> 12 & 63); abyte0[l++] = (byte)(128 | j1 >> 6 & 63); abyte0[l++] = (byte)(128 | j1 & 63); i++; } } else//3 { abyte0[l++] = (byte)(224 | c >> 12); abyte0[l++] = (byte)(128 | c >> 6 & 63); abyte0[l++] = (byte)(128 | c & 63); } } return l; } 1）单字节字符 字符c属于范围[0x0000 , 0x00C8) abyte0[l++] = (byte)c;
byte[]规则： b[0] = #### ####(但不大于0xC8) 2）双字节字符 字符c属于范围[0x00C8 , 0x0800)
abyte0[l++] = (byte)(192 | c >> 6); abyte0[l++] = (byte)(128 | c & 63);
192 <=> 1100 0000 128 <=> 1000 0000 63 <=> 0011 1111 高6位 : b[0] = 110# #### 低6位：b[1] = 10## #### 3）3个字节的字符 字符c属于范围其他情况的补集
abyte0[l++] = (byte)(224 | c >> 12); abyte0[l++] = (byte)(128 | c >> 6 & 63); abyte0[l++] = (byte)(128 | c & 63);
224 <=> 1110 0000 128 <=> 1000 0000 63 <=> 0011 1111 高4位 : b[0] = 1110 #### 中6位：b[1] = 10## #### 低6位：b[2] = 10## #### 4）4个字节的"字符"
Character.isSurrogate(c)
public static boolean isSurrogate(char ch) { return ch >= MIN_SURROGATE && ch < (MAX_SURROGATE + 1); } 可以看到字符c属于范围[0xD800 , 0xDFFF] ，满足这个范围的字符，才算是4个字节的"字符".
/** * The minimum value of a Unicode high-surrogate code unit in the UTF-16 encoding, constant {@code '\u005CuD800'}. * A high-surrogate is also known as a leading-surrogate */ public static final char MIN_HIGH_SURROGATE = '\uD800'; /** * The maximum value of a Unicode high-surrogate code unit in the UTF-16 encoding, constant {@code '\u005CuDBFF'}. * A high-surrogate is also known as a leading-surrogate */ public static final char MAX_HIGH_SURROGATE = '\uDBFF'; /** * The minimum value of a Unicode low-surrogate code unit in the UTF-16 encoding, constant {@code '\u005CuDC00'}. * A low-surrogate is also known as a trailing-surrogate. */ public static final char MIN_LOW_SURROGATE = '\uDC00'; /** * The maximum value of a Unicode low-surrogate code unit in the UTF-16 encoding, constant {@code '\u005CuDFFF'}. * A low-surrogate is also known as a trailing-surrogate */ public static final char MAX_LOW_SURROGATE = '\uDFFF'; /** * The minimum value of a Unicode surrogate code unit in the UTF-16 encoding, constant {@code '\u005CuD800'}. */ public static final char MIN_SURROGATE = MIN_HIGH_SURROGATE; /** * The maximum value of a Unicode surrogate code unit in the UTF-16 encoding, constant {@code '\u005CuDFFF'}. */ public static final char MAX_SURROGATE = MAX_LOW_SURROGATE; /** * The minimum value of a Unicode supplementary code point, constant {@code U+10000}. */ public static final int MIN_SUPPLEMENTARY_CODE_POINT = 0x010000;
以上是下面使用到的一些java.lang.Character源代码常量。
4.1>int j1 = sgp.parse(c, ac, i - 1, k);//4.1
public int parse(char c, char ac[], int i, int j) { if(!$assertionsDisabled && ac[i] != c) throw new AssertionError(); if(Character.isHighSurrogate(c)) { if(j - i < 2) { error = CoderResult.UNDERFLOW; return -1; } char c1 = ac[i + 1]; if(Character.isLowSurrogate(c1)) { character = Character.toCodePoint(c, c1);//a isPair = true; error = null; return character; } else { error = CoderResult.malformedForLength(1); return -1; } } if(Character.isLowSurrogate(c)) { error = CoderResult.malformedForLength(1); return -1; } else {//b character = c; isPair = false; error = null; return character; } } public static boolean isHighSurrogate(char ch) { // Help VM constant-fold; MAX_HIGH_SURROGATE + 1 == MIN_LOW_SURROGATE return ch >= MIN_HIGH_SURROGATE && ch < (MAX_HIGH_SURROGATE + 1); } 高16位范围为：[0xD800,0xDBFF]
public static boolean isLowSurrogate(char ch) { return ch >= MIN_LOW_SURROGATE && ch < (MAX_LOW_SURROGATE + 1); }
低16位范围为：[0xDC00,0xDFFF] b代码块在这里的方法调用是无法进入的，只有是满足高16位范围为：[0xD800,0xDBFF]，低16位范围为：[0xDC00,0xDFFF]，（即本次传入的字符满足[0xD800,0xDBFF]，后跟要有字符，而且字符取值范围[0xDC00,0xDFFF]。
character = Character.toCodePoint(c, c1);//a
public static int toCodePoint(char high, char low) { // Optimized form of: // return ((high - MIN_HIGH_SURROGATE) << 10) // + (low - MIN_LOW_SURROGATE) // + MIN_SUPPLEMENTARY_CODE_POINT; return ((high << 10) + low) + (MIN_SUPPLEMENTARY_CODE_POINT - (MIN_HIGH_SURROGATE << 10) - MIN_LOW_SURROGATE); } 字符c是高位字符，字符c1是低位字符，这里产生的codepoint用于4.3的转化
| high - MIN_HIGH | |||
| low - MIN_LOW | |||
| 0 1 | 0 0 ... ... ... ... 0 | ||
| 8bits | 2bits | 6 bits | 10bits |
4.2>if(j1 < 0)//4.2
if(malformedInputAction() != CodingErrorAction.REPLACE) return -1; abyte0[l++] = replacement()[0];
这里是当出现不符合4个字节字符规则的字符，都会以replacement字符（默认为'?'）进行替代（转换为byte[]后无法转回来相应的unicode码了，用了replacement的unicode码进行替代了）。 4.3>else
} else//4.3 { abyte0[l++] = (byte)(240 | j1 >> 18); abyte0[l++] = (byte)(128 | j1 >> 12 & 63); abyte0[l++] = (byte)(128 | j1 >> 6 & 63); abyte0[l++] = (byte)(128 | j1 & 63); i++; } 240 <=> 1111 0000 128 <=> 1000 0000 63 <=> 0011 1111 高8位 : b[0] = 1111 0### 中高6位：b[1] = 10## #### 低高6位：b[2] = 10## #### 低6位: b[3] = 10## ####