Developing A Custom Apache Nifi Processor JSONThe list of available Apache Nifi processors is extensive, as documented in this post. There is still a need to develop your own to pull data from a database, to process an uncommon file format, or many other unique situations. So to get you started, we will work through a basic processor that takes a json file as input and a json path as a parameter to place into the contents and an attribute. The full source is hosted on Github. Setup. Start by creating a simple maven project in your favorite IDE. Then edit the pom. UTF 8 lt projectxmlnshttp maven. POM4. 0. 0xmlns xsihttp www. XMLSchema instancexsi schema. Locationhttp maven. POM4. 0. 0 http maven. Version 4. 0. Version lt group. Windows 7 Desktop File Organizer. Id rocks. nifilt group. Id lt artifact. Id exampleslt artifact. Hi All, I am writing a RESTful web service which will take a XML file as input and process the same, and save it into database. I am new to RESTful web services. Fix possible race condition when setting IO listeners on an upgraded connection. Enable Tomcat to shutdown cleanly when using sendfile. Apache Commons Io Copy File' title='Apache Commons Io Copy File' />Id lt version 1. SNAPSHOTlt version lt packaging narlt packaging lt properties lt project. Apache Commons Io Copy File' title='Apache Commons Io Copy File' />Jakarta Commons IO Jakarta Commons IOjava. Java Copy File. Java File Copy. Copy File in Java using Java IO Stream, Java NIO FileChannel, Apache Commons IO FileUtils. File, Files. copy, Video tuts. Edit on GitHub Announcements. November 2017 Struts 2. Teletubbies Torrent Download. General Availability 07 September 2017 Struts 2. General Availability 05 September 2017. Encoding UTF 8lt project. Encoding lt maven. Id org. apache. Id lt artifact. Apache Commons Io Copy File' title='Apache Commons Io Copy File' />Id nifi apilt artifact. Id lt version nifi. Id org. apache. Id lt artifact. Id nifi utilslt artifact. Id lt version nifi. Id org. apache. Id lt artifact. Id nifi processor utilslt artifact. Id lt version nifi. Id org. apache. Id lt artifact. Id commons iolt artifact. Id lt version 1. Id com. Id lt artifact. Id json pathlt artifact. Id lt version 1. Id org. Id lt artifact. Id nifi mocklt artifact. Id lt version nifi. Id junitlt group. Id lt artifact. Id junitlt artifact. Id lt version 4. Id org. Id lt artifact. Id nifi nar maven pluginlt artifact. Id lt version 1. Id org. Id lt artifact. Id maven surefire pluginlt artifact. Id lt version 2. This pom. The nifi api is the only other required dependency. The other nifi dependencies are really use full as you will see. The next important piece is telling nifi which classes to load and register. This is done in a single file located at srcmainresourcesMETA INFservicesorg. Processorrocks. nifi. Json. Processor. The JSON Processor. Now that everything is defined and findable by Apache Nifi, lets build a processor. Define a simple java class as defined in the setup process rocks. Json. Processor. Tags are useful for finding your processor in the list of processors in the GUI. So in this case in the search box you could just type json and your processor will be found. The capability description is also displayed in the processor selection box. Nifi. rocks will make future posts on documenting your custom processors. Finally most processors will just extend the Abstract. Processor, for more complicated tasks it may be required to go a level deeper for the Abstract. Session. Factory. Processor. Side. Effect. FreeTagsJSON,NIFI ROCKSCapability. DescriptionFetch value from json path. Json. Processorextends. Abstract. ProcessorNot really interesting stuff here. Properties will hold all a list of all the available properties tha are exposed to the user. Relationships will hold the relationships the processor will use to direct the flow files. For more details on relationships, properties, and components of an Apache Nifi flow please read the offical developer guide. There is plenty of room to expand on custom validators, but there is a large selection of validators in nifi processor utils package. Listlt Property. Descriptor properties private. Setlt Relationship relationships publicstaticfinal. String. MATCHATTRmatch publicstaticfinal. Property. Descriptor. JSONPATHnew. Property. Descriptor. Builder. Json Path. requiredtrue. ValidatorStandard. Validators. NONEMPTYVALIDATOR. Relationship. SUCCESSnew. Relationship. Builder. SUCCESS. descriptionSucces relationship. The init function is called at the start of Apache Nifi. Remember that this is a highly multi threaded environment and be careful what you do in this space. This is why both the list of properties and the set of relationships are set with unmodifiable collections. I put the getters for the properties and relationships here as well. Overridepublicvoidinitfinal. Processor. Initialization. ContextcontextListlt Property. Descriptor propertiesnew. Array. Listlt properties. JSONPATH this. Collections. Listproperties Setlt Relationship relationshipsnew. Hash. Setlt relationships. SUCCESS this. relationshipsCollections. Setrelationships Overridepublic. Setlt Relationship get. Relationshipsreturnrelationships Overridepublic. Listlt Property. Descriptor get. Supported. Property. Descriptorsreturnproperties The on. Trigger method is called when ever a flow file is passed to the processor. For more details on the context and session variables please again refer to the official developer guide. Overridepublicvoidon. TriggerProcess. Contextcontext,Process. Sessionsessionthrows. Process. Exceptionfinal. Atomic. Referencelt String valuenew. Atomic. Referencelt Flow. Fileflowfilesession. Input. Stream. CallbackOverridepublicvoidprocessInput. How To Convert Microsoft Word 2007 Into Pdf. Streaminthrows. IOExceptiontryStringjsonIOUtils. Stringin StringresultJson. Path. readjson,. Exceptionexex. Stack. Trace get. Logger. errorFailed to read json string. Write the results to an attribute. Stringresultsvalue. Emptyflowfilesession. Attributeflowfile,match,results To write the results back out ot flow fileflowfilesession. Output. Stream. CallbackOverridepublicvoidprocessOutput. Streamoutthrows. IOExceptionout. Bytes session. SUCCESS In general you pull the flow file out of session. Read and write to the flow files and add attributes where needed. To work on flow files nifi provides 3 callback interfaces. Input. Stream. Callback For reading the contents of the flow file through a input stream. Using Apache Commons to read the input stream out to a string. Use Json. Path to attempt to read the json and set a value to the pass on. It would normally be best practice in the case of a exception to pass the original flow file to a Error relation point in the case of an exception. Input. Stream. CallbackOverridepublicvoidprocessInput. Streaminthrows. IOExceptiontryStringjsonIOUtils. Stringin StringresultJson. Path. readjson,. Exceptionexex. Stack. Trace get. Logger. errorFailed to read json string. Output. Stream. Callback For writing to a flowfile, this will over write not concatenate. We simply write out the value we recieved in the Input. Stream. Callbackflowfilesession. Output. Stream. CallbackOverridepublicvoidprocessOutput. Streamoutthrows. IOExceptionout. Bytes Stream. Callback This is for both reading and writing to the same flow file. With both the outputstreamcallback and streamcall back remember to assign it back to a flow file. This processor is not in use in the code and could have been. The choice was deliberate to show a way of moving data out of callbacks and back in. Output. Stream. CallbackOverridepublicvoidprocessOutput.