Word Counts Buggy

[naalvarez]

I hate to bother you with another issue! I mentioned this in another thread, and after some testing have noticed patterns that might help fixing it, and also found a workaround for now.

The main issue is that, although the WritingStats code works and updates my Writing Tracker Data sheet, the numbers do not accurately reflect what has been written.

After lots of testing, I realized a few things:

• When "Word Count Mode" is set to 1, so that it tracks {{Fiction}} and {{Nonfiction}} individually, it also counts the tags {{Fiction}} and {{Nonfiction}} as a word. This isn't a problem necessarily, and it doesn't bother me, but it is something to note.
• Also when "Word Count Mode" is set to 1, if a document does NOT have a tag, the code will NOT capture any changes to the document. I thought it might at least add the word count to the Total, but it does not. This happened both in old docs and new docs I created for testing. Again, this doesn't bother me, but it's important to know that you have to add a tag if you want to count Fiction and Nonfiction separately.
• Whether I had "Word Count Mode" set to 1 or 0, the code/data sheet would only capture words written in NEW documents, not existing documents in my "Sandbox" folder.
• A possible work around, at least for now, is to copy and paste all existing documents into new documents. Doing this allows the code to read it, whether you use the tags or not. However, it also counted EVERY word in the new document, including those pasted from the original! So if you do this, you might have a strangely high word count record for the day. ;)

So at least for now I can copy/paste documents into new ones to keep using this Beta version. I'll wait to see if any updates are posted though. :)

Thank you again!

[jamietr1]

The word count has always been slightly off from exact. It has to do with the diff function I'm using (open source). I've noticed it is +/- 30 words, which is okay by my standards. And yes, it does count the tags. I don't do anything to prevent that. Not worth it to me right now.

Also, I never delete anything. I move it to a section of the document at the bottom called "Deleted Scenes" which allows me to get better word counts on days I do a lot of deleting (and avoiding negative word counts if I, say, add 500 new words and delete 700 old ones). The new words are legitimate for that day, even though I am also deleting old words.

I've isolated the other error from earlier and am working on a fix that should be good for everyone, but I may not get to it until later in the week.

[naalvarez]

Ah the not deleting is a good strategy as I keep writing. And it doesn't really bother me if the word count is a little off or if it includes tags. I thought I'd share it in case someone else had that "issue."

But it's definitely not tracking my existing documents. So would you say to just use my workaround and create new documents, then copy/paste existing stories into the new documents? It's a simple enough workaround, my word count will be quite high for today but I'll just disregard that moving forward.

Again, thanks!

[jamietr1]

It will ignore the existing documents in your Sandbox if they are identical to the versions in your Snapshot (Earlier) folder. If you add a few words to an existing document, or remove the version of that document in your Snapshot folder, it should capture the word counts.

Remember, it is really only trying to capture word counts for changes that happen today. If a file didn't change today, it won't get counted, even if it is sitting in your Sandbox folder.

[naalvarez]

Oh I know, but the problem is if I edit an existing document - add words - it doesn't capture those words.

But new documents don't have any issue, so I can just copy/paste to new documents and keep going from there.

[jamietr1]

Okay, l'll open the word count as a separate issue. Good call on the condition where Word Count Mode = 1 and the document has no tag not getting counted. Sure enough, that condition was missing. I've added it and committed that code into the beta branch, so if you get the most recent code, that issue should be fixed.

[naalvarez]

I'm not sure if you updated the code to fix this, but I saw a new WritingStats code and replaced my old one, edited an existing document and ran the code - it looks like it is now capturing edits to existing documents!! Thanks so much for the fix!

[ndandanda]

I am not sure if it's the same issue or another one, but Daily Almanac shows 0 words. My Evernote note "Daily writing for 08/01/2014" created by the script consists of 432 words. However, it's 0 words in "Writing" sheet and consequently in Daily Almanac.

"Writing" sheet does look strange:

7/31/2014 -78 0.0
7/31/2014 0 0.0
7/31/2014 -2 0.2
8/1/2014 0 0.0
8/1/2014 0 10.4

I don't know why it's negative or zero, I actualy did some writing in Sandbox both on 31 and on 1.

But I'll keep trying.

[jamietr1]

Okay, let's see, mine ran fine last night so there are a couple of possibilities I can think of, especially regarding the negative numbers:

1. When you did your writing yesterday, did you do it in the Snapshot folder by accident? Typically, when I've seen negative numbers, it's when I've accidentally edited the files in the Snapshot (formerly "Earlier") folder. When the DIFF command compares the files it assumes the Sandbox is the new and the Snapshot is the old. If you add words to the Snapshot, instead of the Sandbox folder, you can get negative numbers in your word count.
2. But that might not explain the fact that you had 432 words in your summary sent to Evernote. So the next question: are you running code from the most recent commit? There have been a bunch of changes over the last couple of days. As of this moment, the most recent commit?

[ndandanda]

1. No, I didn't.
2. Now I am. Same results.

This is what I wrote today (after deleting and reinstalling):

https://docs.google.com/document/d/1fn4Kg9XPCA5uCFG0xqzL_Cy59i-nODr4iISQFZjj5hQ/edit?usp=sharing

https://docs.google.com/document/d/1B__9ecQM8EY4ACyj2WzJnLvaPJRBEnRd-2c7BCqKFZ0/edit?usp=sharing

This is how my EN note with daily writing looks like: https://www.evernote.com/shard/s1/sh/80c29185-b7c9-4961-a67d-8ac79d13eeaa/80ff8be27f085cbccda9814bcb4760d7

This is the Daily Almanac (I changed the value of Offset Days to 0): https://www.evernote.com/shard/s1/sh/3cb175c7-21ba-4dd3-b246-9168b16fb5d9/b64e11317f3e3e49d745d5a4d4ba0093

Still 0.

I had another wild guess – maybe it didn't work because I do most of my typing in Cyrillic letters (for example https://draftin.com/ counts only Latin characters). But no, this time no Cyrillics – still doesn't work.

There is a typo in the name of the function: getAlmanctext instead of getAlmanactext, but I don't think it matters.

Thank you for your time! I really hope it will work.

[jamietr1]

A couple more questions:

1. The Almanac draws its data from the spreadsheet. So it should be running after the WritingStats script. If you go to your spreadsheet's Writing tab, do you see word counts there for today? Or is it 0? If the latter, that explains why Daily Almanac is reporting 0 because all it's doing is reading from that row.
2. After you did your writing today (the above attachments) did an entry get created in the Writing tab with the proper word count? If not, the problem is the in the WritingStats script and not the Daily Almanac.

Here is what I would suggest:

1. Create one or two more test documents in your Sandbox.
2. Force the WritingStats script to run after you create them. You can do this by opening the script and then from the Run menu, select getDailyWordCount.
3. Once the script has completed, go to the spreadsheet and see if there is a new row with data that reflects the changes you made in step 1.

It's possible that the Daily Almanac was being run before the WritingStats and that is why you were seeing those problems. I ran the same code just now and it is working correctly for me.

If it still doesn't seem to be working (if there are 0 words recorded on the most recent line of the spreadsheet) send me the log output from the script. You can get it by going to the View menu of the WritingStats script, and clicking Logs... copy out what's there and send it along.

[ndandanda]

1. Yes, I ran it after the WritingStats script. The Writing tab now looks like that:

   02/08/2014 0 #NAME? 500
   02/08/2014 0 #NAME? 500 7,2
   02/08/2014 0 #NAME? 500 7,4
   02/08/2014 37 #NAME? 500 7,4
   02/08/2014 0 7 500 12,3
   02/08/2014 7 7 500 12,3

7 is the correct number of words in the third document written today.

This is the log:

[14-08-02 22:52:19:428 CEST] Date not passed in. Using current date: 2014-08-02
[14-08-02 22:52:19:752 CEST] Checking file: doc3...
[14-08-02 22:52:19:753 CEST] -> File was modified today. Getting word count...
[14-08-02 22:52:20:822 CEST] -> Checking file diff for doc3...
[14-08-02 22:52:21:641 CEST] --- FILE DIFF FOLLOWS ---
Can it handle Russian? Don't get excited.
--- FILE DIFF COMPLETE ---
[14-08-02 22:52:21:641 CEST] --- ORIG FILE FOLLOWS ---
Can it handle Russian? Don't get excited.
--- ORIG FILE COMPLETE ---
[14-08-02 22:52:21:988 CEST] -> Backing up doc3 to Earlier...
[14-08-02 22:52:23:588 CEST] -> Backed up original file.
[14-08-02 22:52:23:589 CEST] Checking file: doc1...
[14-08-02 22:52:23:589 CEST] -> File was modified today. Getting word count...
[14-08-02 22:52:25:174 CEST] -> Checking file diff for doc1...
[14-08-02 22:52:25:877 CEST] -> An earlier version of the file exists...
[14-08-02 22:52:25:990 CEST] --- FILE DIFF FOLLOWS ---

--- FILE DIFF COMPLETE ---
[14-08-02 22:52:25:990 CEST] --- ORIG FILE FOLLOWS ---
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test
test

--- ORIG FILE COMPLETE ---
[14-08-02 22:52:26:333 CEST] -> Backing up doc1 to Earlier...
[14-08-02 22:52:27:191 CEST] -> Removed older version to trash...
[14-08-02 22:52:28:339 CEST] -> Backed up original file.
[14-08-02 22:52:28:339 CEST] Checking file: doc2...
[14-08-02 22:52:28:340 CEST] -> File was modified today. Getting word count...
[14-08-02 22:52:29:796 CEST] -> Checking file diff for doc2...
[14-08-02 22:52:30:484 CEST] -> An earlier version of the file exists...
[14-08-02 22:52:30:598 CEST] --- FILE DIFF FOLLOWS ---

--- FILE DIFF COMPLETE ---
[14-08-02 22:52:30:598 CEST] --- ORIG FILE FOLLOWS ---
test test test chicken test cow and chicken test typing test
--- ORIG FILE COMPLETE ---
[14-08-02 22:52:31:188 CEST] -> Backing up doc2 to Earlier...
[14-08-02 22:52:32:009 CEST] -> Removed older version to trash...
[14-08-02 22:52:33:133 CEST] -> Backed up original file.
[14-08-02 22:52:33:495 CEST] -> Sending email

[jamietr1]

@ndandanda: I think I might see a possible problem. The way the Daily Almanac is currently written, it searches the Writing tab for the date in question. When it finds the first instance of that date, it grabs the data and stops. This was by design. My intent was that there would be one record for each day, even days in which no writing took place (all of the numbers would be zero'd out.)

As you indicated above, the first entry for 8/2/2014 in your Writing tab is:

02/08/2014 0 #NAME? 500

Note the 0 words? For now, anyway, there should only ever be one record per day. This is why I have the script run late at night, after I will have done all of my writing for the day. I wasn't looking to capture sessions, just the aggregate for the day.

So, one way of testing this theory would be to have you delete these row:

02/08/2014 0 #NAME? 500
02/08/2014 0 #NAME? 500 7,2
02/08/2014 0 #NAME? 500 7,4
02/08/2014 37 #NAME? 500 7,4
02/08/2014 0 7 500 12,3

and leave behind only this one:

02/08/2014 7 7 500 12,3

which should be the most recent. I suspect that the DA is stopping on that very first entry for the day, the one with 0 words. Once you clear those out and leave in only the most recent, I think the DA will run correctly.

[ndandanda]

No, still 0. Deleted lines as you said and left it alone to run automatically - daily writing note is ok (665 words measured in Evernote), writing sheet is ok (642 words).