Problem of Bulk insert in safari

[ctoph3r]

I have 350 records that need to be insert into an object store, and the size of the 350 records is about 19mb.

here is a function that i use to insert an array list into objectstore:

function saveListObjectStoreValue(thisDB, objectStoreName, list){
        var transaction = thisDB.transaction(objectStoreName,"readwrite");
        var objectStore = transaction.objectStore(objectStoreName);
        for( index in list){
            var req =objectStore.add(list[index]);
            req.onerror = function(event){

            }
            req.onsuccess = function(event){

            }

        }
    }


//here, i am inserting data into an objectstore (contact):
//line code  1
saveListObjectStoreValue(db, 'contact', list); //assume the list contains 100 records and the size is 6mb
//line code 2
saveListObjectStoreValue(db, 'contact', list);   //assume the list contains 100 records and the size is 3mb
//line code 3
saveListObjectStoreValue(db, 'contact', list);   //assume the list contains 100 records  and the size is 5mb
//line code 4
saveListObjectStoreValue(db, 'contact', list);   //assume the list contains 50 records  and the size is 5mb

in the line code 1, after reaching 5mb, safari asks the user , the authorisation to increase the size of web offline storage...

i click on authorise button.. but i have only 90 records(instead of 100) which are stored in the object store 'contact'

in line code 2, the 100 records is correctly inserted..

in line code 3, safari again, asks the authorisation to increase the size of web offline storage, and , i click on authorise button, but i have only got about 20 records inserted in the object store (instead of 100 records... )

in line code 4, the 100 records are correctly inserted...

note 1: i didnt run the 4 lines of code(e.g line code 1,line code 2) at the same time ,but they are run one by one..

note 2 : i have put an alert in the loop, i hv successfully got my 350 alerts...

function saveListObjectStoreValue(thisDB, objectStoreName, list){
        var transaction = thisDB.transaction(objectStoreName,"readwrite");
        var objectStore = transaction.objectStore(objectStoreName);
        for( index in list){
                        alert(index); //my alert!!! <---------------- 
            var req =objectStore.add(list[index]);
            req.onerror = function(event){

            }
            req.onsuccess = function(event){

            }

        }
    }

i think the problem is the safari's popup asking the authorisation to increase the size of web offline storage.. which cancels the following records that still need to be inserted ..

[axemclion]

Do you get errors when you try inserting records ? Or do you get any errors in transactions ? Can you send me the code snippet as a gist - add its link to this issue, so that I can run the file on my browser to see what happens ?

[ctoph3r]

Hi, unfortunately, i am unable to post the codes and data on the internet due to company's policy... in fact, the codes and functionalities are much more complex.. i have just given an example on my previous post..

but, i write a piece of code just for you to test(HTML FILE) --> https://gist.github.com/4130468/9b2cc94cb4d7beb0c49fa47b54e28735a7daf807 .. but i am unable to exactly reproduce the bug on this piece of code..

But if you load my piece of code on safari and ..click on result, it will tell you that there are 0 records in the object store... then if you input Number of records: = 60 , and Size per record: = 60 and click on "run test" button..
you are going to get an alert from safari asking the permission to increase the offline web storage.. but when i click on authorise, it does not insert the records at all.. i have to reclick on run test again..to get my 60 records saved in indexeddb...

is it possible to get the records saved without having to reclick on the "run test" button (when safari asks the permission to increase the size offline storage, of course, we click on authorise)?

[brettz9]

In the case of database errors such as quota errors, we currently have an issue whereby we are setting the transaction's error property to the WebSQL error object rather than creating our own IndexedDB error object.

It is on my to-do list for the next release to report an IndexedDB QuotaExceededError when encountering a WebSQL SQLError.QUOTA_ERR (value 4), a TransactionInactiveError when encountering a WebSQL SQLError.TIMEOUT_ERR (value 7) and any other WebSQL errors as UnknownError.

In the meantime, you can check yourself on the transaction's error property for what should currently be an object containing the code property set to 4 in case of a WebSQL quota error. You can then respond accordingly.

(Note that neither the IndexedDB nor WebSQL error distinguishes between whether the user declined or it is not available.)

Also, or in addition, in master, I have added support to allow configuration of DEFAULT_DB_SIZE (shimIndexedDB.__setConfig('DEFAULT_DB_SIZE', <value>);). We also auto-detect Safari now to allow for it apparently creating larger files (and reaching the quota faster). The normal default is 4 * 1024 * 1024 but for Safari we are now going to 25 * 1024 * 1024 by default. Perhaps setting this to an even larger size (at least for Safari) may help minimize your need for detecting quota errors in the first place.

Hopefully either or both of these approaches may allow you to recover. I'm not sure there is anything else we can do to accommodate, however, so I will close for now. Feel free to open if you have any ideas about how our code could accommodate.

Issue #262 is tracking the next release so when that is ready, you can adjust any error detection code (as described above) to look for the IndexedDB-based properties instead of the WebSQL ones. Issue #41 is tracking the large Safari file size in case there is anything that can be done about it.

[brettz9]

Note also that before the next release, I am hoping to investigate some other reported Safari/iOS issues: https://github.com/axemclion/IndexedDBShim/issues?utf8=%E2%9C%93&q=is%3Aopen%20label%3A%22Bug-environment%20support%22%20-label%3ACSP

[brettz9]

I've fixed master so that it will report IndexedDB-style error objects when SQL errors occur, reporting them all as UnknownError unless the WebSQL error is a quota error, in which case QuotaExceededError is reported.