Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start on negative valence update #302

Merged
merged 44 commits into from
Sep 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
7693289
start on negative valence update
rcoreilly Aug 24, 2023
e00c696
first pass on new pos / neg pvlv code -- simplified vta -- only call …
rcoreilly Aug 25, 2023
982df05
more reorganization, cleanup of variables; building
rcoreilly Aug 25, 2023
5026858
pvlv building, before adding OFCnegUS
rcoreilly Aug 26, 2023
8f0ef55
pvlv building with new OFCnegUS, ACCnegVal layers
rcoreilly Aug 27, 2023
8a8c3c3
pvlv connectivity looks right
rcoreilly Aug 27, 2023
6d3afed
need raw and separately normed USneg vals..
rcoreilly Aug 27, 2023
6117b57
major reorg of pvlv code -- now all CPU-only (still operating on glob…
rcoreilly Aug 28, 2023
90c36ce
fix crash in pvlv -- need better update of context and better update …
rcoreilly Aug 28, 2023
689fb68
start on SetNUSs
rcoreilly Aug 28, 2023
86bdd0f
finally found what was resetting GvHasRew -- logic now working mostly…
rcoreilly Aug 29, 2023
9631c01
consistency is the hobgoblin of coding progress.. but worth it?
rcoreilly Aug 29, 2023
0c89431
moved VTAParams to VTALayer instead of PVLV, runs cycle-by-cycle on C…
rcoreilly Aug 29, 2023
0a44bcf
cleanup of effort max updating and VSGate logic: VS layer sets the ov…
rcoreilly Aug 29, 2023
44c100f
popcode, inhib params updated for us layers -- overall basic pos acq …
rcoreilly Aug 29, 2023
218c8eb
use PVLV NUS values for configuring network, add pvlv monitoring of v…
rcoreilly Aug 29, 2023
d5578e8
updated boa -- needs higher threshold on BLANeg activity to allow cur…
rcoreilly Aug 30, 2023
b6d9485
urgency vs. effort depending on hasgated; update pcore
rcoreilly Aug 30, 2023
54f3708
update rl
rcoreilly Aug 30, 2023
6c13b66
boa wiring and layout fixes
rcoreilly Aug 30, 2023
d145bb8
added NegUSOutcome with threshold for strong negative USs that act to…
rcoreilly Aug 30, 2023
50a5c39
negative conditioning working in pvlv.. key fix in ACh always trigger…
rcoreilly Aug 31, 2023
7ce0671
vspatch learning much better now -- still need to work on extinction …
rcoreilly Aug 31, 2023
027b9a3
pvlv acq and ext looking good -- params tuned
rcoreilly Sep 1, 2023
6906266
better sorting of PV / give up logic
rcoreilly Sep 2, 2023
9ea3f9d
first pass sketch of new give up logic
rcoreilly Sep 2, 2023
1f1a69c
new foraging-based giveup logic in place and cleaned up PVDA logic; a…
rcoreilly Sep 3, 2023
0841103
give up logic fixed - now extinguishing too quickly in bla -- not su…
rcoreilly Sep 4, 2023
dfaa5e7
extinction working -- needed to clear stuff and set HasRew on give up
rcoreilly Sep 4, 2023
b3c6f4e
boa working with new pvlv, exploring params -- sometimes gets stuck a…
rcoreilly Sep 5, 2023
ffc4398
boa params, stats, OpenWts and other config updates
rcoreilly Sep 5, 2023
b4d4da7
new VSPatch learning working much better: intrinsic excitability diff…
rcoreilly Sep 5, 2023
40debdd
good params for boa -- needs more work on pvlv; gpu updated
rcoreilly Sep 6, 2023
9dab0f8
building fixes, and use consistent USpos/negIdx functions to get the …
rcoreilly Sep 6, 2023
298001d
finally figured out how to run Params.SaveAll Good for all examples i…
rcoreilly Sep 7, 2023
9458802
boa test std updated
rcoreilly Sep 7, 2023
f0d28e5
InitActs for GeInt, GiInt -- for passing basic test
rcoreilly Sep 7, 2023
cee182a
fix go.mod and update to emergent 1.4.28
rcoreilly Sep 7, 2023
64fd77d
update standard for boa test
rcoreilly Sep 7, 2023
683bb66
checking for give up rand as cause of diffs between mac and CI
rcoreilly Sep 7, 2023
9430f04
v1.8.16 release -- version.go updated
rcoreilly Sep 7, 2023
d612f1e
another attempt to pass boa test -- exclude factors that differ fm st…
rcoreilly Sep 7, 2023
a68d0bc
finally realized that 'make build' doesn't build the exe -- just test…
rcoreilly Sep 7, 2023
4d53ad0
vspatch dip gain default updated
rcoreilly Sep 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,22 @@ build:
@echo "GO111MODULE = $(value GO111MODULE)"
$(GOBUILD) -v $(DIRS)

build_exe: ./examples/*
@echo "GO111MODULE = $(value GO111MODULE)"
@for f in $^; do \
DIR=$$(basename $${f}) && echo "" && echo "###########################" && \
echo "example: $${DIR} in path: $${f}"; \
cd $${f}; go build -v; cd ../../; \
done

params_good: ./examples/*
@echo "GO111MODULE = $(value GO111MODULE)"
@for f in $^; do \
DIR=$$(basename $${f}) && echo "" && echo "###########################" && \
echo "example: $${DIR} in path: $${f}"; \
cd $${f}; go build -v; ./$${DIR} -no-gui -Params.SaveAll -Params.Good; cd ../../; \
done

test:
@echo "GO111MODULE = $(value GO111MODULE)"
export TEST_GPU=false; $(GOTEST) -v -tags multinet $(DIRS)
Expand Down
2 changes: 1 addition & 1 deletion axon/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ clean:
$(GOCLEAN)

# NOTE: MUST update version number here prior to running 'make release'
VERS=v1.8.15
VERS=v1.8.16
PACKAGE=axon
VERS_DATE=`date -u +%Y-%m-%d\ %H:%M`
GIT_COMMIT=`git rev-parse --short HEAD`
Expand Down
16 changes: 10 additions & 6 deletions axon/act.go
Original file line number Diff line number Diff line change
Expand Up @@ -289,8 +289,8 @@ type DtParams struct {
// [def: 20] [min: 1] time constant for integrating slower long-time-scale averages, such as nrn.ActAvg, Pool.ActsMAvg, ActsPAvg -- computed in NewState when a new input state is present (i.e., not msec but in units of a theta cycle) (tau is roughly how long it takes for value to change significantly) -- set lower for smaller models
LongAvgTau float32 `def:"20" min:"1" desc:"time constant for integrating slower long-time-scale averages, such as nrn.ActAvg, Pool.ActsMAvg, ActsPAvg -- computed in NewState when a new input state is present (i.e., not msec but in units of a theta cycle) (tau is roughly how long it takes for value to change significantly) -- set lower for smaller models"`

// [def: 10] [min: 0] cycle to start updating the SpkMaxCa, SpkMax, GeIntMax values within a theta cycle -- early cycles often reflect prior state
MaxCycStart int32 `def:"10" min:"0" desc:"cycle to start updating the SpkMaxCa, SpkMax, GeIntMax values within a theta cycle -- early cycles often reflect prior state"`
// [def: 10] [min: 0] cycle to start updating the SpkMaxCa, SpkMax values within a theta cycle -- early cycles often reflect prior state
MaxCycStart int32 `def:"10" min:"0" desc:"cycle to start updating the SpkMaxCa, SpkMax values within a theta cycle -- early cycles often reflect prior state"`

// [view: -] nominal rate = Integ / tau
VmDt float32 `view:"-" json:"-" xml:"-" desc:"nominal rate = Integ / tau"`
Expand Down Expand Up @@ -824,6 +824,10 @@ func (ac *ActParams) DecayState(ctx *Context, ni, di uint32, decay, glong, ahp f
AddNrnV(ctx, ni, di, GiNoise, -decay*NrnV(ctx, ni, di, GiNoise))

AddNrnV(ctx, ni, di, GiSyn, -decay*NrnV(ctx, ni, di, GiSyn))

AddNrnV(ctx, ni, di, GeInt, -decay*NrnV(ctx, ni, di, GeInt))
AddNrnV(ctx, ni, di, GiInt, -decay*NrnV(ctx, ni, di, GiInt))
AddNrnV(ctx, ni, di, GeIntNorm, -decay*NrnV(ctx, ni, di, GeIntNorm))
}

AddNrnV(ctx, ni, di, VmDend, -glong*(NrnV(ctx, ni, di, VmDend)-ac.Init.Vm))
Expand Down Expand Up @@ -897,6 +901,9 @@ func (ac *ActParams) InitActs(ctx *Context, ni, di uint32) {
SetNrnV(ctx, ni, di, GiNoise, 0)

SetNrnV(ctx, ni, di, GiSyn, 0)
SetNrnV(ctx, ni, di, GeInt, 0)
SetNrnV(ctx, ni, di, GeIntNorm, 0)
SetNrnV(ctx, ni, di, GiInt, 0)

SetNrnV(ctx, ni, di, MahpN, 0)
SetNrnV(ctx, ni, di, SahpCa, 0)
Expand Down Expand Up @@ -947,7 +954,7 @@ func (ac *ActParams) InitActs(ctx *Context, ni, di uint32) {
}

// InitLongActs initializes longer time-scale activation states in neuron
// (SpkPrv, SpkSt*, ActM, ActP, GeInt, GiInt)
// (SpkPrv, SpkSt*, ActM, ActP)
// Called from InitActs, which is called from InitWts,
// but otherwise not automatically called
// (DecayState is used instead)
Expand All @@ -957,9 +964,6 @@ func (ac *ActParams) InitLongActs(ctx *Context, ni, di uint32) {
SetNrnV(ctx, ni, di, SpkSt2, 0)
SetNrnV(ctx, ni, di, ActM, 0)
SetNrnV(ctx, ni, di, ActP, 0)
SetNrnV(ctx, ni, di, GeInt, 0)
SetNrnV(ctx, ni, di, GiInt, 0)
SetNrnV(ctx, ni, di, GeIntMax, 0)
}

//gosl: start act
Expand Down
40 changes: 23 additions & 17 deletions axon/basic_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
"strings"
"testing"

"github.com/emer/emergent/erand"
"github.com/emer/emergent/etime"
"github.com/emer/emergent/params"
"github.com/emer/emergent/prjn"
Expand Down Expand Up @@ -110,6 +111,9 @@ func newTestNet(ctx *Context, nData int) *Network {
testNet.ConnectLayers(hidLay, outLay, prjn.NewOneToOne(), ForwardPrjn)
testNet.ConnectLayers(outLay, hidLay, prjn.NewOneToOne(), BackPrjn)

testNet.PVLV.SetNUSs(ctx, 4, 3)
testNet.PVLV.Defaults()

testNet.Build(ctx)
ctx.NetIdxs.NData = uint32(nData)
testNet.Defaults()
Expand Down Expand Up @@ -1429,37 +1433,30 @@ func saveToFile(net *Network, t *testing.T) {
func TestGlobalIdxs(t *testing.T) {
ctx := NewContext()
nData := uint32(5)
ctx.PVLV.Drive.NActive = 4
ctx.PVLV.Drive.NNegUSs = 3
net := newTestNet(ctx, int(nData))
pv := &net.PVLV
val := float32(0)

// fmt.Printf("MaxData: %d NActive: %d NNegUSs: %d NetIdxs: GvVTAOff: %d Stride: %d USnegOff: %d DriveOff: %d DriveStride: %d\n", ctx.NetIdxs.MaxData, ctx.PVLV.Drive.NActive, ctx.PVLV.Drive.NNegUSs, ctx.NetIdxs.GvVTAOff, ctx.NetIdxs.GvVTAStride, ctx.NetIdxs.GvUSnegOff, ctx.NetIdxs.GvDriveOff, ctx.NetIdxs.GvDriveStride)
// fmt.Printf("MaxData: %d NActive: %d NNegUSs: %d NetIdxs: USnegOff: %d DriveOff: %d DriveStride: %d\n", ctx.NetIdxs.MaxData, ctx.PVLV.NPosUSs, ctx.PVLV.Drive.NNegUSs, ctx.NetIdxs.GvUSnegOff, ctx.NetIdxs.GvDriveOff, ctx.NetIdxs.GvDriveStride)

for vv := GvRew; vv < GvVtaDA; vv++ {
for vv := GvRew; vv < GvUSneg; vv++ {
for di := uint32(0); di < nData; di++ {
SetGlbV(ctx, di, vv, val)
val += 1
}
}
for vv := GvVtaDA; vv < GvUSneg; vv++ {
for vt := GvVtaRaw; vt < GlobalVTATypeN; vt++ {
for vv := GvUSneg; vv <= GvUSnegRaw; vv++ {
for ui := uint32(0); ui < pv.NNegUSs; ui++ {
for di := uint32(0); di < nData; di++ {
SetGlbVTA(ctx, di, vt, vv, val)
SetGlbUSneg(ctx, di, vv, ui, val)
val += 1
}
}
}
for ui := uint32(0); ui < ctx.PVLV.Drive.NNegUSs; ui++ {
for di := uint32(0); di < nData; di++ {
SetGlbUSneg(ctx, di, ui, val)
val += 1
}
}
for vv := GvDrives; vv < GlobalVarsN; vv++ {
for ui := uint32(0); ui < ctx.PVLV.Drive.NActive; ui++ {
for ui := uint32(0); ui < pv.NPosUSs; ui++ {
for di := uint32(0); di < nData; di++ {
SetGlbDrvV(ctx, di, ui, vv, val)
SetGlbUSposV(ctx, di, vv, ui, val)
val += 1
}
}
Expand All @@ -1477,8 +1474,6 @@ func TestGlobalIdxs(t *testing.T) {
func TestSendGatherIdxs(t *testing.T) {
ctx := NewContext()
nData := uint32(3)
ctx.PVLV.Drive.NActive = 4
ctx.PVLV.Drive.NNegUSs = 3
net := newTestNet(ctx, int(nData))

maxDel := net.MaxDelay + 1
Expand Down Expand Up @@ -1597,6 +1592,17 @@ func TestSendGatherIdxs(t *testing.T) {
}
}

func TestPVLVGiveUp(t *testing.T) {
t.Skip("")
gp := &GiveUpParams{}
gp.Defaults()
rnd := erand.NewGlobalRand()
for v := float32(-1.0); v <= float32(1); v += 0.01 {
p, b := gp.Prob(v, rnd)
fmt.Printf("%g\tp: %g\tb: %v\n", v, p, b)
}
}

/* todo: fixme
func TestSWtInit(t *testing.T) {
pj := &PrjnParams{}
Expand Down
Loading