Index is screw up #4089

martingg88 · 2015-05-06T03:38:33Z

it try to screw up my index when i migrated from 2.0.8 to 2.1-rc2.
FYI. indexes are exist for 2.0.8 but it's rebuild and deleted at 2.1-rc2 due to following issue. any idea? this issue will only affect for hash index only. furtheremore, the save button is disabled when create index at studio.

2015-05-06 11:32:55:455 WARNING {db=sport} Cannot load index 'account.language' from storage (rid=#1:1012): rebuilt it from scratch [OIndexNotUnique]{db=sport} Cannot rebuild index 'account.language' from storage (rid=#1:1012) because 'java.lang.IllegalArgumentException: File with id 3560 not found in WOW Cache'. The index will be removed in configuration
com.orientechnologies.orient.core.exception.OStorageException: File with name account.language.hib does not exist in storage sport

swimmesberger · 2015-05-08T10:18:54Z

Having a similar problem with 2.1-rc2 here. When orientdb is trying to rebuild the database via WAL I get following exception:

SEVERE: {db=system} Exception during storage data restore.
com.orientechnologies.orient.core.exception.OStorageException: File with name Mapping.localId.irs does not exist in storage system
    at com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.openFile(OWOWCache.java:317)
    at com.orientechnologies.orient.core.index.hashindex.local.cache.O2QCache.openFile(O2QCache.java:184)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreAtomicUnit(OAbstractPaginatedStorage.java:2293)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFrom(OAbstractPaginatedStorage.java:2237)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFromFullCheckPoint(OAbstractPaginatedStorage.java:2198)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFromCheckPoint(OAbstractPaginatedStorage.java:2187)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFromWAL(OAbstractPaginatedStorage.java:2138)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreIfNeeded(OAbstractPaginatedStorage.java:1491)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:165)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:223)
    at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire(OPartitionedDatabasePool.java:287)
    at 
com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage 'system' with mode=rw
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:210)
    at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:223)
    at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire(OPartitionedDatabasePool.java:287)
    at 
Caused by: com.orientechnologies.orient.core.exception.OStorageException: File with name Mapping.localId.irs does not exist in storage system
    at com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.openFile(OWOWCache.java:317)
    at com.orientechnologies.orient.core.index.hashindex.local.cache.O2QCache.openFile(O2QCache.java:184)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreAtomicUnit(OAbstractPaginatedStorage.java:2293)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFrom(OAbstractPaginatedStorage.java:2237)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFromFullCheckPoint(OAbstractPaginatedStorage.java:2198)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFromCheckPoint(OAbstractPaginatedStorage.java:2187)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFromWAL(OAbstractPaginatedStorage.java:2138)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreIfNeeded(OAbstractPaginatedStorage.java:1491)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:165)
    ... 11 more

It's not only occuring with an upgrade form an older version it also happens when I just kill the process and so the WAL is used on next startup.

andrii0lomakin · 2015-05-08T10:43:46Z

Sorry I do not get it. You killed the server and trying to update db ?

swimmesberger · 2015-05-08T10:47:40Z

The initial poster said its happening on an upgrade from 2.0.8 to 2.1-rc2 but I can also reproduce it with a fresh 2.1-rc2 database with some indices. When you forcefully stop the database (e.g. killing the process) the WAL is used on next startup and when it tries to rebuild the indices on that startup the error occured for me.

andrii0lomakin · 2015-05-08T10:50:34Z

We support binary compatibility if you do not kill db process but correctly close it.
About first post. @martingg88 Could you provide example of db, so I will try to open it too ?

martingg88 · 2015-05-08T11:13:51Z

you can download db at this link
https://drive.google.com/file/d/0B4HzIyaEAQ5WTV9CNkliS01CMzQ/view?pli=1

martingg88 · 2015-05-13T02:50:24Z

any update?

machak · 2015-05-20T16:11:06Z

have seen same issue on (2-1.SNAPSHOT).

"We support binary compatibility if you do not kill db process but correctly close it."
so, in case of server crash, data is pretty much corrupted...no?

martingg88 · 2015-05-20T16:40:05Z

we don't have this issue before 2.1 version.

andrii0lomakin · 2015-05-20T16:52:08Z

@machak if you crash server then you should restart it and then close it again. So your data will not be corrupted.

andrii0lomakin · 2015-05-21T12:52:29Z

Fixed, coul dyou try now ?

swimmesberger · 2015-05-26T13:45:40Z

With 2.1-rc3 it works now for me :)

martingg88 · 2015-05-26T15:55:40Z

yes. it work now

daniellawry · 2015-12-08T18:18:32Z

May be related... I'm seeing the following trace when querying the graph db using the JAVA API

java.lang.IllegalArgumentException: File with id 0 not found in WOW Cache
    at com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.cacheFileContent(OWOWCache.java:899)
    at com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.load(OWOWCache.java:417)
    at com.orientechnologies.orient.core.index.hashindex.local.cache.OReadWriteDiskCache.updateCache(OReadWriteDiskCache.java:676)
    at com.orientechnologies.orient.core.index.hashindex.local.cache.OReadWriteDiskCache.doLoad(OReadWriteDiskCache.java:302)
    at com.orientechnologies.orient.core.index.hashindex.local.cache.OReadWriteDiskCache.load(OReadWriteDiskCache.java:270)
    at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.loadPageEntry(OLocalHashTable.java:2085)
    at com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.get(OLocalHashTable.java:404)
    at com.orientechnologies.orient.core.index.engine.OHashTableIndexEngine.get(OHashTableIndexEngine.java:155)
    at com.orientechnologies.orient.core.index.OIndexOneValue.get(OIndexOneValue.java:58)
    at com.orientechnologies.orient.core.index.OIndexOneValue.get(OIndexOneValue.java:45)
    at com.orientechnologies.orient.core.index.OIndexAbstractDelegate.get(OIndexAbstractDelegate.java:59)
    at com.orientechnologies.orient.core.index.OIndexTxAwareOneValue.get(OIndexTxAwareOneValue.java:233)
    at com.orientechnologies.orient.core.index.OIndexTxAwareOneValue.get(OIndexTxAwareOneValue.java:41)
    at com.orientechnologies.orient.core.sql.operator.OQueryOperatorEquals.executeIndexQuery(OQueryOperatorEquals.java:129)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:1525)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:757)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:193)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:440)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:422)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:393)
    at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:64)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:1176)
    at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:1165)
    at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:63)
    at com.tinkerpop.blueprints.impls.orient.OrientGraphCommand.execute(OrientGraphCommand.java:48)

First occurred in 2.1.2. I tried upgrading to 2.1.7 but the issue remained.

I have done both proper restarts and shutdowns of the orientdb server and have tried to rebuild all indexes but no luck.

Possible regression?

andrii0lomakin · 2015-12-09T07:09:30Z

Hi Daniel,
Could you send me your database ?
We had issues with atomic operations management before 2.1.5 in hash index
I think that is your case.
But we have to check .

вт, 8 Дек 2015, 20:18, Daniel Lawry notifications@github.com:

May be related... I'm seeing the following trace when querying the graph
db using the JAVA API

java.lang.IllegalArgumentException: File with id 0 not found in WOW Cache
at
com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.cacheFileContent(OWOWCache.java:899)
at
com.orientechnologies.orient.core.index.hashindex.local.cache.OWOWCache.load(OWOWCache.java:417)
at
com.orientechnologies.orient.core.index.hashindex.local.cache.OReadWriteDiskCache.updateCache(OReadWriteDiskCache.java:676)
at
com.orientechnologies.orient.core.index.hashindex.local.cache.OReadWriteDiskCache.doLoad(OReadWriteDiskCache.java:302)
at
com.orientechnologies.orient.core.index.hashindex.local.cache.OReadWriteDiskCache.load(OReadWriteDiskCache.java:270)
at
com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.loadPageEntry(OLocalHashTable.java:2085)
at
com.orientechnologies.orient.core.index.hashindex.local.OLocalHashTable.get(OLocalHashTable.java:404)
at
com.orientechnologies.orient.core.index.engine.OHashTableIndexEngine.get(OHashTableIndexEngine.java:155)
at
com.orientechnologies.orient.core.index.OIndexOneValue.get(OIndexOneValue.java:58)
at
com.orientechnologies.orient.core.index.OIndexOneValue.get(OIndexOneValue.java:45)
at
com.orientechnologies.orient.core.index.OIndexAbstractDelegate.get(OIndexAbstractDelegate.java:59)
at
com.orientechnologies.orient.core.index.OIndexTxAwareOneValue.get(OIndexTxAwareOneValue.java:233)
at
com.orientechnologies.orient.core.index.OIndexTxAwareOneValue.get(OIndexTxAwareOneValue.java:41)
at
com.orientechnologies.orient.core.sql.operator.OQueryOperatorEquals.executeIndexQuery(OQueryOperatorEquals.java:129)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:1525)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:757)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:193)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:440)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:422)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:393)
at
com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:64)
at
com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:1176)
at
com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:1165)
at
com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:63)
at
com.tinkerpop.blueprints.impls.orient.OrientGraphCommand.execute(OrientGraphCommand.java:48)

First occurred in 2.1.2. I tried upgrading to 2.1.7 but the issue remained.

I have done both proper restarts and shutdowns of the orientdb server and
have tried to rebuild all indexes but no luck.

Possible regression?

—
Reply to this email directly or view it on GitHub
#4089 (comment)
.

Best regards,
Andrey Lomakin, R&D lead.
OrientDB Ltd

twitter:@Andrey_Lomakin linkedin:https://ua.linkedin.com/in/andreylomakin

lvca assigned andrii0lomakin May 20, 2015

andrii0lomakin added in progress bug labels May 20, 2015

andrii0lomakin modified the milestones: 2.1 Enterprise GA, 2.1 GA May 20, 2015

andrii0lomakin added a commit that referenced this issue May 21, 2015

Issue #4089 is fixed.

c44bbca

andrii0lomakin closed this as completed May 21, 2015

andrii0lomakin removed the in progress label May 21, 2015

lvca modified the milestones: 2.1-rc3, 2.1 GA May 21, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Index is screw up #4089

Index is screw up #4089

martingg88 commented May 6, 2015

swimmesberger commented May 8, 2015

andrii0lomakin commented May 8, 2015

swimmesberger commented May 8, 2015

andrii0lomakin commented May 8, 2015

martingg88 commented May 8, 2015

martingg88 commented May 13, 2015

machak commented May 20, 2015

martingg88 commented May 20, 2015

andrii0lomakin commented May 20, 2015

andrii0lomakin commented May 21, 2015

swimmesberger commented May 26, 2015

martingg88 commented May 26, 2015

daniellawry commented Dec 8, 2015

andrii0lomakin commented Dec 9, 2015

Index is screw up #4089

Index is screw up #4089

Comments

martingg88 commented May 6, 2015

swimmesberger commented May 8, 2015

andrii0lomakin commented May 8, 2015

swimmesberger commented May 8, 2015

andrii0lomakin commented May 8, 2015

martingg88 commented May 8, 2015

martingg88 commented May 13, 2015

machak commented May 20, 2015

martingg88 commented May 20, 2015

andrii0lomakin commented May 20, 2015

andrii0lomakin commented May 21, 2015

swimmesberger commented May 26, 2015

martingg88 commented May 26, 2015

daniellawry commented Dec 8, 2015

andrii0lomakin commented Dec 9, 2015