Find Number of fields in a file

To find the number of fields in a TSV file just do the following:

First calculate the number of tabs:

$ head -1 /tmp/file.txt |  egrep -o -E $'\t' | wc -l

The number of fields is number of tabs separating the fields + 1

16 + 1 = 17


Now read this

Setting up Camus - LinkedIn’s Kafka to HDFS pipeline

Few days ago I started tinkering with Camus to evaluate its use for dumping raw data from Kafka=>HDFS. This blog post will cover my experience and first impressions with setting up a Camus pipeline. Overall I found Camus was easy to... Continue →