Reproducing results from 2019 target tracking paper

[jturner314]

Your 2019 target tracking paper, "Learning Q-network for Active Information Acquisition," is really interesting work. I'd like to build on and cite it, so I'm trying to reproduce the results. I've had success running the code, but I've been unsuccessful in getting results that match Fig. 2.

The version of the code that appears to most closely match the description in the paper is a19dd2a. I've started with the simplest case: the "ADFQ, TS" method for the "Single Target - Empty Environment" problem. I'm running the following in Google Colaboratory; I believe this matches all the available information provided in the paper:

%tensorflow_version 1.x
!git clone https://github.com/coco66/ADFQ.git
!cd ADFQ; git checkout -b reproduce; git reset --hard a19dd2a2e5c012e3bd7f6129c7beab51313b6f39; cd ..
!pip install filterpy

python_paths = ['ADFQ', 'ADFQ/classic_adfq', 'ADFQ/deep_adfq']
import sys
sys.path.extend(python_paths)
import os
os.environ['PYTHONPATH'] += ':' + ':'.join(python_paths)

!echo -e '\n' | python ADFQ/deep_adfq/run_tracking.py --nb_train_steps 3000 --act_policy bayesian --map empty

The versions are:

• Python 3.6.9
• TensorFlow 1.15.2
• filterpy 1.4.5

This is the image batch_0/mean_nlogdetcov.png in the results directory:

and this is the moving average with window 4 of mean_nlogdetcov:

which doesn't match Fig. 2 of the paper.

What am I missing? Was a different version of the code used to generate the results in the paper? Are there other options/parameters that I need to explicitly set to match those used for the paper?

[coco66]

Hi Jim, Thanks for your interest in my paper. You may be using a different environmental parameter setup. Are you using METADATA_v1 in ttenv/ttenv/metadata.py ? Also, what is your noise matrix of the target model? Best, Heejin

…

On Jun 17, 2020, at 4:46 PM, Jim Turner ***@***.***> wrote: Your 2019 target tracking paper, "Learning Q-network for Active Information Acquisition <https://arxiv.org/abs/1910.10754>," is really interesting work. I'd like to build on and cite it, so I'm trying to reproduce the results. I've had success running the code, but I've been unsuccessful in getting results that match Fig. 2. The version of the code that appears to most closely match the description in the paper is a19dd2a <a19dd2a>. I've started with the simplest case: the "ADFQ, TS" method for the "Single Target - Empty Environment" problem. I'm running the following in Google Colaboratory <https://colab.research.google.com/>; I believe this matches all the available information provided in the paper: %tensorflow_version 1.x !git clone https://github.com/coco66/ADFQ.git !cd ADFQ; git checkout -b reproduce; git reset --hard a19dd2a; cd .. !pip install filterpy python_paths = ['ADFQ', 'ADFQ/classic_adfq', 'ADFQ/deep_adfq'] import sys sys.path.extend(python_paths) import os os.environ['PYTHONPATH'] += ':' + ':'.join(python_paths) !echo -e '\n' | python ADFQ/deep_adfq/run_tracking.py --nb_train_steps 3000 --act_policy bayesian --map empty The versions are: Python 3.6.9 TensorFlow 1.15.2 filterpy 1.4.5 This is the image batch_0/mean_nlogdetcov.png in the results directory: <https://user-images.githubusercontent.com/1836850/84946258-66981200-b0b6-11ea-9dbb-0bc8b036058b.png> and this is the moving average with window 4 of mean_nlogdetcov: <https://user-images.githubusercontent.com/1836850/84946275-6bf55c80-b0b6-11ea-8179-54dad8be9321.png> which doesn't match Fig. 2 of the paper. What am I missing? Was a different version of the code used to generate the results in the paper? Are there other options/parameters that I need to explicitly set to match those used for the paper? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7T3FAJX3H2ZXXF3H5CLN3RXETQXANCNFSM4OA6PWHA>.

[jturner314]

Hi Heejin,

Thanks for responding so quickly.

I've been testing with python ADFQ/deep_adfq/run_tracking.py --nb_train_steps 3000 --act_policy bayesian --map empty with version a19dd2a of the ADFQ repository, which was before the target tracking environment was moved into the separate ttenv GitHub repository. This version uses METADATA_v0, which seems to match the description in the paper more closely than METADATA_v1. (init_distance is 8m, as described in the paper). It uses the TargetTracking-v1 environment, which computes the noise covariance matrix of the target model like this. I've been testing with the "empty" map because the paper says that the single-target empty domain case is 100m×100m, which seems to match empty.yaml.

I chose this version and setup just because it seemed to most closely match the description in the paper. Should I be using a different version or environment to reproduce the results in the paper?

Thanks,
Jim

[coco66]

HI Jim, I have been developing/testing my codes in my private repo and updated the public repo occasionally. I am able to replicate the result from my private repo with the version around the time of the paper submission (the results attached below), but there have been a lot of changes since then and I cannot quickly figure out the major differences. I currently have other works but I will check my codes with details for the next couple of days and get back to you. Thanks again for reaching out to me with the issue. Best, Heejin

…

On Jun 17, 2020, at 8:49 PM, Jim Turner ***@***.***> wrote: Hi Heejin, Thanks for responding so quickly. I've been testing with python ADFQ/deep_adfq/run_tracking.py --nb_train_steps 3000 --act_policy bayesian --map empty with version a19dd2a <a19dd2a> of the ADFQ repository, which was before the target tracking environment was moved into the separate ttenv GitHub repository. This version uses METADATA_v0 <https://github.com/coco66/ADFQ/blob/a19dd2a2e5c012e3bd7f6129c7beab51313b6f39/envs/target_tracking/metadata.py#L2-L18>, which seems to match the description in the paper more closely than METADATA_v1. (init_distance is 8m, as described in the paper). It uses the TargetTracking-v1 environment, which computes the noise covariance matrix of the target model like this <https://github.com/coco66/ADFQ/blob/a19dd2a2e5c012e3bd7f6129c7beab51313b6f39/envs/target_tracking/target_tracking.py#L237-L239>. I've been testing with the "empty" map because the paper says that the single-target empty domain case is 100m×100m, which seems to match empty.yaml <https://github.com/coco66/ADFQ/blob/a19dd2a2e5c012e3bd7f6129c7beab51313b6f39/envs/maps/empty.yaml>. I chose this version and setup just because it seemed to most closely match the description in the paper. Should I be using a different version or environment to reproduce the results in the paper? Thanks, Jim — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7T3FALYSAJZIOAOW5XLRTRXFQAXANCNFSM4OA6PWHA>.