logo
down
shadow

How to avro binary encode my json string to a byte array?


How to avro binary encode my json string to a byte array?

By : user2955263
Date : November 22 2020, 10:38 AM
I wish this help you It seems to be working at least. It can be simplified though: the loops are useless since having more than one object would result in invalid JSON. Also, it could be a good idea to avoid unnecessary parsing of the schema by preparsing it.
Here's my version:
code :
public class AvroTest {

    private static final String JSON = "{" + "\"name\":\"Frank\"," + "\"age\":47" + "}";
    private static final Schema SCHEMA = new Schema.Parser().parse("{ \"type\":\"record\", \"namespace\":\"foo\", \"name\":\"Person\", \"fields\":[ { \"name\":\"name\", \"type\":\"string\" }, { \"name\":\"age\", \"type\":\"int\" } ] }");

    public static void main(String[] args) throws IOException {
        byte[] data = jsonToAvro(JSON, SCHEMA);

        String jsonString = avroToJson(data, SCHEMA);
        System.out.println(jsonString);
    }

    /**
     * Convert JSON to avro binary array.
     * 
     * @param json
     * @param schema
     * @return
     * @throws IOException
     */
    public static byte[] jsonToAvro(String json, Schema schema) throws IOException {
        DatumReader<Object> reader = new GenericDatumReader<>(schema);
        GenericDatumWriter<Object> writer = new GenericDatumWriter<>(schema);
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        Decoder decoder = DecoderFactory.get().jsonDecoder(schema, json);
        Encoder encoder = EncoderFactory.get().binaryEncoder(output, null);
        Object datum = reader.read(null, decoder);
        writer.write(datum, encoder);
        encoder.flush();
        return output.toByteArray();
    }

    /**
     * Convert Avro binary byte array back to JSON String.
     * 
     * @param avro
     * @param schema
     * @return
     * @throws IOException
     */
    public static String avroToJson(byte[] avro, Schema schema) throws IOException {
        boolean pretty = false;
        GenericDatumReader<Object> reader = new GenericDatumReader<>(schema);
        DatumWriter<Object> writer = new GenericDatumWriter<>(schema);
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        JsonEncoder encoder = EncoderFactory.get().jsonEncoder(schema, output, pretty);
        Decoder decoder = DecoderFactory.get().binaryDecoder(avro, null);
        Object datum = reader.read(null, decoder);
        writer.write(datum, encoder);
        encoder.flush();
        output.flush();
        return new String(output.toByteArray(), "UTF-8");
    }
}


Share : facebook icon twitter icon
How to encode/decode Kafka messages using Avro binary encoder?

How to encode/decode Kafka messages using Avro binary encoder?


By : user3822364
Date : March 29 2020, 07:55 AM
like below fixes the issue I finally remembered to ask the Kafka mailing list and got the following as an answer, which worked perfectly.
Is it possible to encode an integer to byte array to be binary comparable

Is it possible to encode an integer to byte array to be binary comparable


By : Azeem Tariq
Date : March 29 2020, 07:55 AM
this will help This is in C#, but you should be able to do something similar to this in your language of choice, it'll give you the correct sort order for positive as well as negative integers. Note that you need to use an unsigned byte type to sort correctly.
code :
static byte[] EncodeToBytes(int val)
{
    val -= int.MinValue;
    var result = new byte[4];
    for (int i = 0; i < 4; i++, val <<= 8)
        result[i] = (byte)(val >> 24);
    return result;
}

static int DecodeFromBytes(byte[] val)
{
    var result = 0;
    for (int i = 0; i < 4; i++)
        result = (result << 8) | (val[i] & 0xff);
    return result + int.MinValue;
}
Encode a Clojure byte array to JSON

Encode a Clojure byte array to JSON


By : MAYANK MOHAN
Date : March 29 2020, 07:55 AM
To fix this issue While Clojure will resolve literal symbols containing . as the Java class named by the symbol, you can't specify array classes as Clojure literals because the reader interprets the [ as a token signifying the beginning of a vector. As suggested by this thread, the most concise way to get the byte-array class would be:
code :
 (add-encoder (Class/forName "[B") encode-seq)
How to Avro Binary encode the JSON String using Apache Avro?

How to Avro Binary encode the JSON String using Apache Avro?


By : Trung Le
Date : March 29 2020, 07:55 AM
To fix this issue I think OP is correct. This will write Avro records themselves without the schema that would be present if this were an Avro data file.
Here's a couple examples within Avro itself (useful if you are working with files.
Encode an object with Avro to a byte array in Python

Encode an object with Avro to a byte array in Python


By : Prakash Bethapudi
Date : March 29 2020, 07:55 AM
wish helps you Your question helped me figure things out, so thanks. Here's a simple python example based on the python example in the docs:
Related Posts Related Posts :
  • How to write generic Java API without method Overloading
  • Java Serialization / Deserialization of an ArrayList works only on first program execution
  • Conway's Game of Life's rules dont apply properly
  • call a java method when Click on a html button without using javascript
  • Why does SSLSocketFactory.createSocket returns a Socket instead of SSLSocket?
  • How to schedule a task in Tomcat
  • compare two pdf files (approach) using java
  • What is the use of allowCoreThreadTimeout( ) in ThreadPoolExecutor?
  • Describe the Strategy Design Pattern in Java?
  • MongoDB too many open connections
  • Issue with generate JAXB classes with value constructors
  • Moving from one frame to another in java swing
  • Get a thread by Id
  • how to open front camera and back camera at same time?
  • Messed up with Java Declaration
  • Extracting chart in EXcel as image using java
  • How to get latitude and longitude in Android phone?
  • Do GC pauses and kill -STOP produce the same behavior?
  • Android HTML.ImageGetter Images resized
  • The way to instantiate map<String, List<String>> in Java
  • Java Making objects move while buttons held
  • Androidx Navigation View - `setNavigationItemSelectedListener` Doesn't Work
  • after compiling code JFrame does not appear
  • Netbeans project, JFrames ran from the main JFrame close all JFrames in project
  • Square Retrofit Client: How to enable/disable followRedirects? How to intercept redirect URL?
  • Java Circular Reference - Can’t Compile
  • Mouse click on specific time JAVA
  • android 4 onCreateDialog return type incompatible when overriding
  • Java Implementation of Oracle's DBMS_UTILITY.GET_HASH_VALUE Function
  • javac cannot be run, and furthermore does not seem installed
  • Function to obtain a thread given its ID
  • Java File path to Executable has spaces
  • Minimum Oracle Java VM heap memory setting?
  • Elegant way to create one of a large number of classes
  • Java fx binding styles
  • Summing weights based on string in large file
  • Android ListView setOnItemClickListener not working
  • How is the Java Bootstrap Classloader loaded?
  • Check object is blank
  • Add image between text in JEditorPane
  • Java POI Excel creating new column and new rows
  • NullPointerException when managing bean with Spring+JSF(Primefaces)
  • Content autocomplete in spring tool suite 4
  • Super Noob Help ~ MouseEvent/Frame
  • Making one project with dependencies stand-alone in Eclipse (Java)
  • Jsoup- getting certain attributes from website
  • MongoDB, how to make a link between collections with JAVA code
  • Overriding an abstract method means class abstract class or not in java?
  • Bukkit teleport - nullPointerException
  • How to check if a Float variable is negative in java
  • Servlet: Cannot forward after response has been committed
  • sql query to select a specific cell in excel
  • How to make new picture when mouse dragged in java
  • Use Constants interface in GWT Project
  • Setting size of a JPanel in GroupLayout
  • Construct a map from two equal size arrays
  • Implementing Read-Write Locks with Double-Checked Locking
  • Disable NO_BUTTON from JOptionPane Java if a condition is true
  • UDP multiple socket
  • Android Threading: This Handler class should be static or leaks might occur
  • shadow
    Privacy Policy - Terms - Contact Us © ourworld-yourmove.org