How to compile Apache Zeppelin with Spark 1.6

zeppelinhqdefaultRecently I found Apache Zeppelin, an Apache Incubator project that seems to bring a new paradox into the data science game, and other areas.

Something I’ve really like about Zeppelin is the ease of interaction with spark, I use the spark-shell all the time, but it’s tedious having to re-evaluate commands that I previously inputted, Zeppelin fixes this problem. It let’s me go back and forth across the script that I’m building on spark which is nice.

At time of writing the latest release of Zeppelin is 0.5.6, which comes bundled with Spark 1.4.1 but for reasons I want to use Spark 1.6 so in order to build Zeppelin with Spark 1.6 you are going to have to build it from the source.

1.- Download the latest stable source code from Zeppelin’s download page:

https://zeppelin.incubator.apache.org/download.html

2.- untar

tar -zxvf zeppelin-0.5.6-incubating.tgz

3.- compile with support for spark 1.6

mvn clean package -Pspark-1.6 -Dspark.version=1.6.0 -Dhadoop.version=2.6.0-cdh5.4.8 -Phadoop-2.6 -Pyarn -Ppyspark -Pvendor-repo -DskipTests

For more information on what other parameters you can tweak, checkout Zepellin’s Readme file

Solr FieldType class to type

Solr, you are great but we need a list of mappings between your classes and your types to understand everyone’s examples of you.

Here’s a small list I’ve compile for Solr 5.3

 

Class Type
org.apache.solr.schema.TrieLongField
org.apache.solr.schema.TrieDateField tdate
org.apache.solr.schema.BoolField boolean
org.apache.solr.schema.TextField text_general
org.apache.solr.schema.StrField strings

I hope this is useful for everyone landing on this artcile

How to fix HBase 0.94.x not starting after downgrade

hbase_logo-470x140After downgrading my HBase installation from HBase 1.1.2 (currently stable at time of writing) to HBase 0.94.27, for purposes of compatibility with Gora I found myself unable to run HBase with multiple errors, after a couple of hours of debugging I found that the solution is to simply delete all the files in the hbase.rootdir which I specified in my hbase-site.xml

so, if your hbase-site.xml has the following properties

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>

...
<property>
<name>hbase.rootdir</name>
<value>file:///home/hbuser/HBASE/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value> file:///home/hbuser/HBASE/hbase</value>
</property>
...

</configuration>

 

just cd to the previous directory and drop the hbase folder and recreated

cd /home/hbuser/HBASE
rm -Rf hbase
mkdir hbase

after this, just start hbase with start-hbase.sh and eveything should be fine.

Related Error Stacks:

org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
at org.apache.hadoop.hbase.util.Bytes.toBytes(Bytes.java:442)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid