msck repair table for custom partition names

msck repair table is used to add partitions that exist in HDFS but not in the hive metastore.

However, it expects the partitioned field name to be included in the folder structure:
year=2015
|
|_month=3
|
|_day=5

Notice the partition name prefixed with the partition. This is necessary. msck repair table wont work if you have data in the following directory structure:
2015
|
|_3
|
|_5

This is kind of a pain. The only solution is to use alter table add partition with location.

ALTER TABLE test ADD PARTITION (year=2015,month=03,day=05) location ‘hdfs:///cool/folder/with/data’;

 
36
Kudos
 
36
Kudos

Now read this

How to read ISO 8601

ISO 8601 is a format of expressing a date with timezone information. I used to get confused after looking at dates like “2014-10-07T16:11:24-07:00”. Ok so you can tell it is 7th October 2014 and 4:24 PM. The -07:00 tells us the timezone... Continue →