Crash will happen when ramIndex is empty in loadFromIndex

The fix is here:
https://github.com/senseidb/zoie/commit/3e5392e706d36fbbe3540948fcbaaa4214ca0e5c?w=1

Generally it won't happen, but it happened.

In RealtimeIndexDataLoader, processBatch() it checks eventCount.

if (eventCount > 0)
{
long t1=System.currentTimeMillis();
try
{
if(readOnlyMemIndex != null){
_luceneDataLoader.loadFromIndex(readOnlyMemIndex);
}
}

But eventCount > 0 doesn't mean the index is not empty, since in LuceneIndexDataLoader.java

@Override
public void consume(Collection<DataEvent<ZoieIndexable>> events) throws ZoieException {

if (indexable == null || indexable.isSkip()) {
continue;
}

Some event is skipped.

In our case all events are skipped, these events are from sensei to update water mark.

In DefaultStreamingIndexingManager of sensei

@Override
public void consume(Collection<proj.zoie.api.DataConsumer.DataEvent<JSONObject>> data) throws ZoieException
{

JSONObject markerObj = new FastJSONObject();
//markerObj.put(_senseiSchema.getSkipField(), "true");
markerObj.put(SenseiSchema.EVENT_TYPE_FIELD, SenseiSchema.EVENT_TYPE_SKIP);
markerObj.put(_uidField, 0L); // Add a dummy uid
partDataSet.add(new DataEvent<JSONObject>(markerObj, _currentVersion));

So if all events are skipped then RAMIndex maybe empty, and in BaseSearchIndex.java

public void loadFromIndex(BaseSearchIndex<R> index) throws IOException
{

// hao: open readOnly ram index reader
ZoieIndexReader<R> reader = index.openIndexReader();
if(reader == null) return;

Directory dir = reader.directory();
IndexWriter writer = null;
try
{
writer = openIndexWriter(null,null);

OpenIndexWriter is not run and index.directory file is not created.

And in LuceneIndexDataLoader.java

public void loadFromIndex(RAMSearchIndex<R> ramIndex) throws ZoieException
{
try
{
String newVersion = idx.getVersion() == null ? ramIndex.getVersion() : (_versionComparator.compare(idx.getVersion(), ramIndex.getVersion()) < 0 ? ramIndex.getVersion(): idx.getVersion());
idx.setVersion(newVersion);

In DefaultDirectoryManager.java setVersion will crash, since sig is NULL