3,008,816 events, 1,474,912 push events, 2,343,491 commit messages, 180,772,003 characters
fucking hacky ass shit
Signed-off-by: Kris Nóva kris@nivenly.com
Update region.class.php
fucking stupid fuckin stupid fuck fuck hate ajax hate databases they should all burn
async_unary_call: add a Destroy method to the async reader interface.
Before this commit, we specialized std::default_delete for
ClientAsyncResponseReaderInterface to do nothing. This means that, despite the
fact that async stub methods return
std::unique_ptr<ClientAsyncResponseReaderInterface>, it's impossible to have a
subclass of that interface that is allocated on the heap or whose destructor has
side effects without a hack that manages ownership outside of the unique_ptr
returned by the stub.
This was introduced by
dd36b153,
which introduced the specialization for grpc::ClientAsyncResponseReader
itself, and
e8a61d63b,
which did so for the interface. Neither of them left any explanation
whatsoever. :-/
I think the intent is a combination of the following facts:
-
In production uses,
ClientAsyncResponseReaderHelper::Createallocates theClientAsyncResponseReaderon an arena. -
Although it's technically non-trivial, the
ClientAsyncResponseReaderdestructor doesn't actually have any important side effects. -
Therefore it's not necessary to run the destructor, and it's not correct to use
deleteto free the backing memory.
From this point of view, the original commit (dd36b153) is technically
correct: std::unique_ptr<ClientAsyncResponseReader> doesn't actually need to
do anything. But:
-
Of course the objects may be deleted other ways. If I have a
std::unique_ptr<ClientAsyncResponseReaderInterface>(i.e. without a specific deleter type), it would be natural for me to assume that I can dodelete p.release()to get the same effect as destroying the unique pointer. But that is not true, and doing so will be undefined behavior.So from this point of view
dd36b153is not correct, or at least makes for a surprising API. -
ClientAsyncResponseReaderis not the only subclass of the interface. The interface's destructor is virtual, so other subclasses may have destructors with side effects. However they are never actually run! This is a huge surprise: it took me hours yesterday to figure out that I wasn't actually crazy:delete p.release()really was working differently than just letting astd::unique_ptr pgo out of scope.So from this point of view
e8a61d63is also not correct.
This is not a theoretical concern: I've found multiple hacks in user code working around this with a grumbling tone. For example, see this write-up in the cloud bigtable library that uses the phrase "terrible, horrible, no good, very bad hack".
This commit makes it possible for subclasses to opt into reasonable behavior by
defining an override for Destroy that calls delete this. We change the
specialization to call Destroy in order to make this possible. There is no
behavior change in existing code. I will follow up separately with a change to
the base implementation of Destroy, making the reasonable behavior opt-out
instead`, which will be a behavior change for any subclass that depends on the
surprising default.
async_unary_call: add a Destroy method to the async reader interface.
Before this commit, we specialized std::default_delete for
ClientAsyncResponseReaderInterface to do nothing. This means that, despite the
fact that async stub methods return
std::unique_ptr<ClientAsyncResponseReaderInterface>, it's impossible to have a
subclass of that interface that is allocated on the heap or whose destructor has
side effects without a hack that manages ownership outside of the unique_ptr
returned by the stub.
This was introduced by dd36b153, which introduced the specialization for
grpc::ClientAsyncResponseReader itself, and e8a61d63b, which did so
for the interface. Neither of them left any explanation. :-/
I think the intent is a combination of the following facts:
-
In production uses,
ClientAsyncResponseReaderHelper::Createallocates theClientAsyncResponseReaderon an arena. -
Although it's technically non-trivial, the
ClientAsyncResponseReaderdestructor doesn't actually have any important side effects. -
Therefore it's not necessary to run the destructor, and it's not correct to use
deleteto free the backing memory.
From this point of view, the original commit (dd36b153) is technically
correct: std::unique_ptr<ClientAsyncResponseReader> doesn't actually need to
do anything. But:
-
Of course the objects may be deleted other ways. If I have a
std::unique_ptr<ClientAsyncResponseReaderInterface>(i.e. without a specific deleter type), it would be natural for me to assume that I can dodelete p.release()to get the same effect as destroying the unique pointer. But that is not true, and doing so will be undefined behavior.So from this point of view
dd36b153is not correct, or at least makes for a surprising API. -
ClientAsyncResponseReaderis not the only subclass of the interface. The interface's destructor is virtual, so other subclasses may have destructors with side effects. However they are never actually run! This is a huge surprise: it took me hours yesterday to figure out that I wasn't actually crazy:delete p.release()really was working differently than just letting astd::unique_ptr pgo out of scope.So from this point of view
e8a61d63is also not correct.
This is not a theoretical concern: I've found multiple hacks in user code working around this with a grumbling tone. For example, see in the cloud bigtable library that uses the phrase "terrible, horrible, no good, very bad hack":
https://github.com/googleapis/google-cloud-cpp/blob/74de20227371119ea8b8c788d37ae4e24be8ed5d/google/cloud/bigtable/testing/mock_response_reader.h#L77-L106
This commit makes it possible for subclasses to opt into reasonable behavior by
defining an override for Destroy that calls delete this. We change the
specialization to call Destroy in order to make this possible. There is no
behavior change in existing code. I will follow up separately with a change to
the base implementation of Destroy, making the reasonable behavior opt-out
instead`, which will be a behavior change for any subclass that depends on the
surprising default.
Finish the switch to rubyzip
We switched from zipruby to rubyzip in 2018
(commit d65d5a3166edd2b152070207536b6f21a4e9b41d)
However, we forgot to change the gemspec in the proces (oopsie), so
apps that use this gem are still installing zipruby, which hasn't been
touched in 10 years, and doesn't compile today without magic compiler
flags.
This finishes that upgrade.
I chose version 1.3.0 because our main app happens to be using another
gem which requires rubyzip, and it specifies < 2.0.0, and I want to
limit the rabbit hole i'm digging for myself here. But ideally we'd move
this to the latest version.
RD-2273 queued exec inte-tests: handle changed timezones (#3027)
- .all() for the get-queued query
explicitly run the query right there, rather than implicitly via iterating over it. Mostly doesn't matter, helps with debugging. Might well be explicit
- UTCDateTime: pass a tz-naive datetime back to the db
We get a datetime from the db, we add the string 'Z' to it, making a string representation that does contain a timezone; so going back, we must mirror that and strip it back.
Also, note that this can be either a string (in case it was retrieved from the db) or a datetime object (in case it was just assigned), so handle both. This is because we for some reason love strings rather than actual types. But we'll get to eliminating silly stuff like .get_formatted_timestamp in due time. Not yet. For now, handle both.
- EventsAlternativeTimezoneTest: set the db tz back
these tests change the database's tz, so it would be nice if they cleaned up after themselves and set it back.
- flake
github actions can suck my fucking nuts
i'll just compile my shit myself. FUCK
BACKPORT: signal: add pidfd_send_signal() syscall
The kill() syscall operates on process identifiers (pid). After a process has exited its pid can be reused by another process. If a caller sends a signal to a reused pid it will end up signaling the wrong process. This issue has often surfaced and there has been a push to address this problem [1].
This patch uses file descriptors (fd) from proc/ as stable handles on struct pid. Even if a pid is recycled the handle will not change. The fd can be used to send signals to the process it refers to. Thus, the new syscall pidfd_send_signal() is introduced to solve this problem. Instead of pids it operates on process fds (pidfd).
/* prototype and argument /* long pidfd_send_signal(int pidfd, int sig, siginfo_t *info, unsigned int flags);
/* syscall number 424 */ The syscall number was chosen to be 424 to align with Arnd's rework in his y2038 to minimize merge conflicts (cf. [25]).
In addition to the pidfd and signal argument it takes an additional siginfo_t and flags argument. If the siginfo_t argument is NULL then pidfd_send_signal() is equivalent to kill(, ). If it is not NULL pidfd_send_signal() is equivalent to rt_sigqueueinfo(). The flags argument is added to allow for future extensions of this syscall. It currently needs to be passed as 0. Failing to do so will cause EINVAL.
/* pidfd_send_signal() replaces multiple pid-based syscalls */ The pidfd_send_signal() syscall currently takes on the job of rt_sigqueueinfo(2) and parts of the functionality of kill(2), Namely, when a positive pid is passed to kill(2). It will however be possible to also replace tgkill(2) and rt_tgsigqueueinfo(2) if this syscall is extended.
/* sending signals to threads (tid) and process groups (pgid) */ Specifically, the pidfd_send_signal() syscall does currently not operate on process groups or threads. This is left for future extensions. In order to extend the syscall to allow sending signal to threads and process groups appropriately named flags (e.g. PIDFD_TYPE_PGID, and PIDFD_TYPE_TID) should be added. This implies that the flags argument will determine what is signaled and not the file descriptor itself. Put in other words, grouping in this api is a property of the flags argument not a property of the file descriptor (cf. [13]). Clarification for this has been requested by Eric (cf. [19]). When appropriate extensions through the flags argument are added then pidfd_send_signal() can additionally replace the part of kill(2) which operates on process groups as well as the tgkill(2) and rt_tgsigqueueinfo(2) syscalls. How such an extension could be implemented has been very roughly sketched in [14], [15], and [16]. However, this should not be taken as a commitment to a particular implementation. There might be better ways to do it. Right now this is intentionally left out to keep this patchset as simple as possible (cf. [4]).
/* naming */ The syscall had various names throughout iterations of this patchset:
- procfd_signal()
- procfd_send_signal()
- taskfd_send_signal() In the last round of reviews it was pointed out that given that if the flags argument decides the scope of the signal instead of different types of fds it might make sense to either settle for "procfd_" or "pidfd_" as prefix. The community was willing to accept either (cf. [17] and [18]). Given that one developer expressed strong preference for the "pidfd_" prefix (cf. [13]) and with other developers less opinionated about the name we should settle for "pidfd_" to avoid further bikeshedding.
The "_send_signal" suffix was chosen to reflect the fact that the syscall takes on the job of multiple syscalls. It is therefore intentional that the name is not reminiscent of neither kill(2) nor rt_sigqueueinfo(2). Not the fomer because it might imply that pidfd_send_signal() is a replacement for kill(2), and not the latter because it is a hassle to remember the correct spelling - especially for non-native speakers - and because it is not descriptive enough of what the syscall actually does. The name "pidfd_send_signal" makes it very clear that its job is to send signals.
/* zombies */ Zombies can be signaled just as any other process. No special error will be reported since a zombie state is an unreliable state (cf. [3]). However, this can be added as an extension through the @flags argument if the need ever arises.
/* cross-namespace signals */ The patch currently enforces that the signaler and signalee either are in the same pid namespace or that the signaler's pid namespace is an ancestor of the signalee's pid namespace. This is done for the sake of simplicity and because it is unclear to what values certain members of struct siginfo_t would need to be set to (cf. [5], [6]).
/* compat syscalls */ It became clear that we would like to avoid adding compat syscalls (cf. [7]). The compat syscall handling is now done in kernel/signal.c itself by adding __copy_siginfo_from_user_generic() which lets us avoid compat syscalls (cf. [8]). It should be noted that the addition of __copy_siginfo_from_user_any() is caused by a bug in the original implementation of rt_sigqueueinfo(2) (cf. 12). With upcoming rework for syscall handling things might improve significantly (cf. [11]) and __copy_siginfo_from_user_any() will not gain any additional callers.
/* testing */ This patch was tested on x64 and x86.
/* userspace usage */ An asciinema recording for the basic functionality can be found under [9]. With this patch a process can be killed via:
#define _GNU_SOURCE #include <errno.h> #include <fcntl.h> #include <signal.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/stat.h> #include <sys/syscall.h> #include <sys/types.h> #include <unistd.h>
static inline int do_pidfd_send_signal(int pidfd, int sig, siginfo_t *info, unsigned int flags) { #ifdef __NR_pidfd_send_signal return syscall(__NR_pidfd_send_signal, pidfd, sig, info, flags); #else return -ENOSYS; #endif }
int main(int argc, char *argv[]) { int fd, ret, saved_errno, sig;
if (argc < 3)
exit(EXIT_FAILURE);
fd = open(argv[1], O_DIRECTORY | O_CLOEXEC);
if (fd < 0) {
printf("%s - Failed to open \"%s\"\n", strerror(errno), argv[1]);
exit(EXIT_FAILURE);
}
sig = atoi(argv[2]);
printf("Sending signal %d to process %s\n", sig, argv[1]);
ret = do_pidfd_send_signal(fd, sig, NULL, 0);
saved_errno = errno;
close(fd);
errno = saved_errno;
if (ret < 0) {
printf("%s - Failed to send signal %d to process %s\n",
strerror(errno), sig, argv[1]);
exit(EXIT_FAILURE);
}
exit(EXIT_SUCCESS);
}
/* Q&A
- Given that it seems the same questions get asked again by people who are
- late to the party it makes sense to add a Q&A section to the commit
- message so it's hopefully easier to avoid duplicate threads.
- For the sake of progress please consider these arguments settled unless
- there is a new point that desperately needs to be addressed. Please make
- sure to check the links to the threads in this commit message whether
- this has not already been covered. */ Q-01: (Florian Weimer [20], Andrew Morton [21]) What happens when the target process has exited? A-01: Sending the signal will fail with ESRCH (cf. [22]).
Q-02: (Andrew Morton [21]) Is the task_struct pinned by the fd? A-02: No. A reference to struct pid is kept. struct pid - as far as I understand - was created exactly for the reason to not require to pin struct task_struct (cf. [22]).
Q-03: (Andrew Morton [21]) Does the entire procfs directory remain visible? Just one entry within it? A-03: The same thing that happens right now when you hold a file descriptor to /proc/ open (cf. [22]).
Q-04: (Andrew Morton [21]) Does the pid remain reserved? A-04: No. This patchset guarantees a stable handle not that pids are not recycled (cf. [22]).
Q-05: (Andrew Morton [21]) Do attempts to signal that fd return errors? A-05: See {Q,A}-01.
Q-06: (Andrew Morton [22]) Is there a cleaner way of obtaining the fd? Another syscall perhaps. A-06: Userspace can already trivially retrieve file descriptors from procfs so this is something that we will need to support anyway. Hence, there's no immediate need to add another syscalls just to make pidfd_send_signal() not dependent on the presence of procfs. However, adding a syscalls to get such file descriptors is planned for a future patchset (cf. [22]).
Q-07: (Andrew Morton [21] and others) This fd-for-a-process sounds like a handy thing and people may well think up other uses for it in the future, probably unrelated to signals. Are the code and the interface designed to permit such future applications? A-07: Yes (cf. [22]).
Q-08: (Andrew Morton [21] and others) Now I think about it, why a new syscall? This thing is looking rather like an ioctl? A-08: This has been extensively discussed. It was agreed that a syscall is preferred for a variety or reasons. Here are just a few taken from prior threads. Syscalls are safer than ioctl()s especially when signaling to fds. Processes are a core kernel concept so a syscall seems more appropriate. The layout of the syscall with its four arguments would require the addition of a custom struct for the ioctl() thereby causing at least the same amount or even more complexity for userspace than a simple syscall. The new syscall will replace multiple other pid-based syscalls (see description above). The file-descriptors-for-processes concept introduced with this syscall will be extended with other syscalls in the future. See also [22], [23] and various other threads already linked in here.
Q-09: (Florian Weimer [24]) What happens if you use the new interface with an O_PATH descriptor? A-09: pidfds opened as O_PATH fds cannot be used to send signals to a process (cf. [2]). Signaling processes through pidfds is the equivalent of writing to a file. Thus, this is not an operation that operates "purely at the file descriptor level" as required by the open(2) manpage. See also [4].
/* References */ [1]: https://lore.kernel.org/lkml/20181029221037.87724-1-dancol@google.com/ [2]: https://lore.kernel.org/lkml/874lbtjvtd.fsf@oldenburg2.str.redhat.com/ [3]: https://lore.kernel.org/lkml/20181204132604.aspfupwjgjx6fhva@brauner.io/ [4]: https://lore.kernel.org/lkml/20181203180224.fkvw4kajtbvru2ku@brauner.io/ [5]: https://lore.kernel.org/lkml/20181121213946.GA10795@mail.hallyn.com/ [6]: https://lore.kernel.org/lkml/20181120103111.etlqp7zop34v6nv4@brauner.io/ [7]: https://lore.kernel.org/lkml/36323361-90BD-41AF-AB5B-EE0D7BA02C21@amacapital.net/ [8]: https://lore.kernel.org/lkml/87tvjxp8pc.fsf@xmission.com/ [9]: https://asciinema.org/a/IQjuCHew6bnq1cr78yuMv16cy [11]: https://lore.kernel.org/lkml/F53D6D38-3521-4C20-9034-5AF447DF62FF@amacapital.net/ [12]: https://lore.kernel.org/lkml/87zhtjn8ck.fsf@xmission.com/ [13]: https://lore.kernel.org/lkml/871s6u9z6u.fsf@xmission.com/ [14]: https://lore.kernel.org/lkml/20181206231742.xxi4ghn24z4h2qki@brauner.io/ [15]: https://lore.kernel.org/lkml/20181207003124.GA11160@mail.hallyn.com/ [16]: https://lore.kernel.org/lkml/20181207015423.4miorx43l3qhppfz@brauner.io/ [17]: https://lore.kernel.org/lkml/CAGXu5jL8PciZAXvOvCeCU3wKUEB_dU-O3q0tDw4uB_ojMvDEew@mail.gmail.com/ [18]: https://lore.kernel.org/lkml/20181206222746.GB9224@mail.hallyn.com/ [19]: https://lore.kernel.org/lkml/20181208054059.19813-1-christian@brauner.io/ [20]: https://lore.kernel.org/lkml/8736rebl9s.fsf@oldenburg.str.redhat.com/ [21]: https://lore.kernel.org/lkml/20181228152012.dbf0508c2508138efc5f2bbe@linux-foundation.org/ [22]: https://lore.kernel.org/lkml/20181228233725.722tdfgijxcssg76@brauner.io/ [23]: https://lwn.net/Articles/773459/ [24]: https://lore.kernel.org/lkml/8736rebl9s.fsf@oldenburg.str.redhat.com/ [25]: https://lore.kernel.org/lkml/CAK8P3a0ej9NcJM8wXNPbcGUyOUZYX+VLoDFdbenW3s3114oQZw@mail.gmail.com/
Cc: "Eric W. Biederman" ebiederm@xmission.com Cc: Jann Horn jannh@google.com Cc: Andy Lutomirsky luto@kernel.org Cc: Andrew Morton akpm@linux-foundation.org Cc: Oleg Nesterov oleg@redhat.com Cc: Al Viro viro@zeniv.linux.org.uk Cc: Florian Weimer fweimer@redhat.com Signed-off-by: Christian Brauner christian@brauner.io Reviewed-by: Tycho Andersen tycho@tycho.ws Reviewed-by: Kees Cook keescook@chromium.org Reviewed-by: David Howells dhowells@redhat.com Acked-by: Arnd Bergmann arnd@arndb.de Acked-by: Thomas Gleixner tglx@linutronix.de Acked-by: Serge Hallyn serge@hallyn.com Acked-by: Aleksa Sarai cyphar@cyphar.com
(cherry picked from commit 3eb39f47934f9d5a3027fe00d906a45fe3a15fad)
Conflicts: arch/x86/entry/syscalls/syscall_32.tbl - trivial manual merge arch/x86/entry/syscalls/syscall_64.tbl - trivial manual merge include/linux/proc_fs.h - trivial manual merge include/linux/syscalls.h - trivial manual merge include/uapi/asm-generic/unistd.h - trivial manual merge kernel/signal.c - struct kernel_siginfo does not exist in 4.14 kernel/sys_ni.c - cond_syscall is used instead of COND_SYSCALL arch/x86/entry/syscalls/syscall_32.tbl arch/x86/entry/syscalls/syscall_64.tbl
(1. manual merges because of 4.14 differences 2. change prepare_kill_siginfo() to use struct siginfo instead of kernel_siginfo 3. use copy_from_user() instead of copy_siginfo_from_user() in copy_siginfo_from_user_any() 4. replaced COND_SYSCALL with cond_syscall 5. Removed __ia32_sys_pidfd_send_signal in arch/x86/entry/syscalls/syscall_32.tbl. 6. Replaced __x64_sys_pidfd_send_signal with sys_pidfd_send_signal in arch/x86/entry/syscalls/syscall_64.tbl.)
Bug: 135608568 Test: test program using syscall(__NR_pidfd_send_signal,..) to send SIGKILL Change-Id: I34da11c63ac8cafb0353d9af24c820cef519ec27 Signed-off-by: Suren Baghdasaryan surenb@google.com
"10:05am. I had some inspiration last night on long term credit assignment. I said that the answer must be to predict them, but I started to imagine, what if the input to the net is a one hot vector? Then I could just accumulate the gradients and pass them on when the neuron is active.
This is it. Without a doubt this is how long term credit assignment should be done.
https://arxiv.org/abs/1809.03702 Sparse Attentive Backtracking - Temporal Credit Assignment Through Reminding
I saw this paper mentioned in the Optimizing Agent Behavior over Long Time Scales by Transporting Value paper, and just its title was enough to catch my attention. I read it a bit, and this paragraph caught my attention.
Humans have a remarkable ability to remember events from the distant past which are associated with the current mental state (Ciaramelli et al., 2008). Most experimental and theoretical analyses of memory have focused on understanding the deliberate route to memory formation and recall. But automatic reminding—when memories pop into one’s head—can have a potent influence on cognition. Reminding is normally triggered by contextual features present at the moment of retrieval which match distinctive features of the memory being recalled (Berntsen et al., 2013; Wharton et al., 1996), and can occur more often following unexpected events (Read & Cesa, 1991). Thus, an individual’s current state of understanding can trigger reminding of a past state. Reminding can provide distracting sources of irrelevant information (Forbus et al., 1995; Novick, 1988), but it can also serve a useful computational role in ongoing cognition by providing information essential to decision making (Benjamin & Ross, 2010).
In this paper, we identify another possible role of reminding: to perform credit assignment across long time spans. Consider the following scenario. As you drive down the highway, you hear an unusual popping sound. You think nothing of it until you stop for gas and realize that one of your tires has deflated, at which point you are suddenly reminded of the pop. The reminding event helps determine the cause of your flat tire, and probably leads to synaptic changes by which a future pop sound while driving would be processed differently. Credit assignment is critical in machine learning. Back-propagation is fundamentally performing credit assignment. Although some progress has beenmade toward credit-assignment mechanisms that are functionally equivalent to back-propagation (Lee et al., 2014; Scellier & Bengio, 2016; Whittington & Bogacz, 2017), it remains very unclear how the equivalent of back-propagation through time, used to train recurrent neural networks (RNNs), could be implemented by brains. Here we explore the hypothesis that an associative reminding process could play an important role in propagating credit across long time spans, also known as the problem of learning long-term dependencies in RNNs, i.e., of learning to exploit statistical dependencies between events and variables which occur temporally far from each other.
I completely forgot that long term memories are strengthened by reminders in the human brain!
Without a doubt, the simple idea I had is correct. Rather than using a separate net to predict the gradients given the inputs, it would be far better to just accumulate and take advantage of sparsity.
I only read the paper for a bit as it was 1am when I got this idea, I went to bed soon afterwards. Let me do it right now.
http://proceedings.mlr.press/v130/lamb21a.html Neural Function Modules with Sparse Arguments - A Dynamic Approach to Integrating Information across Layers
Also, this caught my attention.
The key contribution of our work is to combine attention, sparsity, top-down and bottom-up feedback, in a flexible algorithm which, as we show, improves the results in standard classification, out-of-domain generalization, generative modeling, and learning representations in the context of reinforcement learning.
In the context of RL. That seems up my alley.
At any rate, long term credit assignment seemed like a huge deal, but looking at it now, it is pretty much solved as far as I am concerned.
10:45am. The SAB paper is too complicated. No, it is not as simple as my own idea. Let me read the other paper.
6/15. These papers are so underwhelming. I am not interested in minor improvements such as these.
11:10am. Forget it, let me take a break here. I don't know, maybe the idea I had is bad. It would reinforce wrong memories in many cases. Just forget it. It is programming time for me. The community can work things out on its own.
12:20pm. Done with chores. Breakfast is next.
I had some inspiration and I just figured out how to do exact gradient propagation in temporal hierarchies when the timescales of different modules are arbitrary.
Let me go back to the one shot idea of input vectors. This is merely the starting point.
-
In such an where inputs only have a single active element at any given time, only the active element would have its gradient propagated.
-
But even if I accumulate the gradients for the next time the element gets activated, the gradient path on the lower layer might not be the same. So the credit assignment would have variance. This scheme is a dead end.
What I started thinking about is going to my earlier idea of predicting the gradients. In order to improve the accuracy of such prediction, I thought about - what if I passed extra information just for the sake of the backward pass. At first I thought these should be ordinary features, but then I realized something interesting.
The input at each feedforward layer is just input * grad. Inputs I can keep around in memory without large impact. It would not be more expensive than regular backprop.
But grad multipliers could be precalculated.
I can just pretend that the gradient coming from above is 1, and then calculate the grad coeficient.
12:35pm. Actually, no wait. Isn't this the same as keeping around the trace?
Actually it is.
Forget that. An even better idea would in fact be to keep around the trace, but use sparsity to filter out the irrelevant timesteps for the upper layers.
Yeah, backprop is not wrong. It is just that you can't expect to keep everything around when attending to multiple timescales. What is needed is a filtering mechanism.
I do not really know what it would look like.
12:40pm. Ok, it is not important at this jucture. This level of insight might not be enough to invent the thing outright and resolve long term credit assignment issues, but it is enough to allow me to predict where the innovations will come from.
Let me have breakfast here."
Taller 3 archivos subidos
Hello darkness, my old friend I've come to talk with you again Because a vision softly creeping Left its seeds while I was sleeping And the vision that was planted in my brain Still remains Within the sound of silence
I think I may have unbelievably solved the ridiculous hybrid user override/median cost calculation for the cost by cancer view. Man, what a pain in the ass that was.
Added New Faction (Harmonizing Musicians)
• Added New Faction (Harmonizing Musicians): "...Realize I'm merely an imitation of humans.", "Gonna Jump Now and Be Free", "I'm Addicted to the Madness", "Living in the Shadows", Amp up the Volume, Backstage Meltdown, Band Equipment, Cocaine Induced Inspiration, ?, Digital Diva, Ecdysiast Singer, Inspiring Message, ?, Label Manager, Lust, Musician's Laptop, Pitch Correction Implant, Record Sales, Street Performer, Sway of the Music, Tour Deadlines, Vocaloid Voice Actress, and Vocal Training. • Added Anger, Bouncer, Devoted Mother, Double or Nothing, Farmer, Fear, Happiness, Information Sniffer, Internet Search, Office Worker, Reconnaissance Satellite, Sadness, School Boy, School Girl, Scry, Secretary, Security Guard, and Traffic Cop. • Hana has been renamed to Blossom (except on her origin card where she has been renamed to Hana Shinutsu.) • Field Generator is now a Generator Asset. • Changed Tradeable to only be activateable once each turn (but it is no longer restricted to deployment phases).
Revert "add first hajf of my favorite song"
This reverts commit fd83e9eee20098cb9c3f5b61e0a921ce9d081005.
I was born in 1970, outskirts of Moscow1 Went wild since I was in diapers I swore too much at the age of Saw girl's c*nt for the first time in the kindergarten And later on in the school with a crappy uniform2, fights and glue3 The things that made me stronger I used to steal cash from the dressing rooms4 I took up smoking at the age of 8 And got laid for the first time at 11 Then I bailed on parents Began to hang out at the dump Boots and a military jacket Brezhnev died5, I began lifting at the gym, Bought boxing gloves, which Were stolen at school. Didn't work too well for Vitjok and Kolja Floor, the blood stains, got filed a case Had burgled a makeshift barrack at night, got props, time flows... Dodged military due to kidney disease6, got drunk and rough up a cop Then moved to the chick's place in Kuryanovo With my fellows we began to rob the cooperators On Rizhskaya9, killed a gypsy woman, that's how it ended up. I found myself on the federal wanted list, bought a car Got hooked on meth Just Butyrka helped, thanks to Kerya. The court a year later, the sentence, 15 years The north, midge, prison settlement11, fierce faces I'm a loose cannon, escaped the courtroom with some goof We fled to the forest, I'm eating his meat, the freight, hitchhiking... Moscow, lay down low New passport, new job, new car Addressing issues, lots of cocaine I got married, backing the banks, settled down in the mansion on Taganka My son was born, my mother died, I went on Cuba on vacation for a half a year...
"1:20pm. Done with breakfast. Let me chill a little.
Attentional mechanisms are definitely the secret behind long term credit assignment. It would be just too much work to scrap all the backprop rules and come up with completely new things. It is much more likely the case that keeping traces around selectively is the secret behind the brain capability in this domain.
1:35pm. Let me catch up with Hagure Idol and I will get some programming done. It is not my job to worry about attention mechanisms and the exact way of making long term credit assignment work.
So what if the hints are there? Making a new algorithm from such sparse info is a lot harder than solving a murder mystery. I am just wasting my time. Eventually, it will get figured out what the brain does. Until then, I should just focus on my own niche which is programming.
1:40pm. I've caught up to the Lord of Mysteries manhua, now let me do the same for Hagure Idol. I am a few chapters behind. After that the Kumo thread. After that I'll try to get some programming done today. Half a month is nearly over and I barely did a thing. I can qualify the last two weeks as a vacation.
2:55pm. Pfft...how did I waste this much time?
Let me start. Unless I want to become a researcher and dedicate myself to studying math and doing experiments in order to write papers, I should focus on the task at hand.
I am at a huge disadvantage not having powerful enough hardware and not being able to use the brain's internals as a reference. It bothers me. All of this does bother me.
For the past 3 weeks, this has been the biggest hurdle for me to cross which is coping with my inadequacy. No matter how powerful my imagination is, I can't imagine opening the skull and peering at the brain inside. I can't reason out the correctness of float using programs in advance.
Right now I am obsessing about my gradient ideas propagation ideas. I need to leave that out of mind.
Eventually the mysteries will be solved and I will be able to chase the superhuman with the tools that I need.
3:05pm. The ML researchers probably could not program as well that I do, but that won't help me unless I actually do it.
I hate ML down to my core. I read papers, but it only opens up more questions, rarely does it give me answers! I am so prideful, but it wants me to show humility. It points a gun at me, threatens to kill me, and asks me to keep calm. I hate it.
3:10pm. Right now I should start work on the uniform player, and then the linear one, but my mind is too much in disarray. I'd rather think about the papers that I've read than do programming work.
Sigh...and I had such good pace a few days ago. I gave into the urge to read papers, and this is the result. But it is fine. I won't read the same thing again. In the past few weeks my focus keeps switching targets, eventually I'll lock on to the original goal.
Let me take a break just to focus on the task at hand. Maybe I should step away from the screen for a bit. Let me do that.
4:50pm. I need to look up if any citations for Generative Minimization Networks came out. In the ML thread, one person said that no image samples is a red flag. Nope, nothing.
Generative Adversarial Networks are Special Cases of Artificial Curiosity (1990) and also Closely Related to Predictability Minimization (1991)
Right now I am reading this paper by Schmidhubber instead.
The NIPS 2014 GAN paper [20] states that PM differs from GANs in the sense that PM is NOT based on a minimax game with a value function that one agent seeks to maximize and the other seeks to minimise. It states that for GANs "the competition between the networks is the sole training criterion, and is sufficient on its own to train the network," while PM "is only a regularizer that encourages the hidden units of a neural network to be statistically independent while they accomplish some other task; it is not a primary training criterion" [20]. But this claim is incorrect, since PM is indeed a pure minimax game, too, e.g., [82], Equation 2. There is no "other task." In particular, PM was also trained [64, 67, 68, 82, 86, 72] (also on images [82, 86]) such that "the competition between the networks is the sole training criterion, and is sufficient on its own to train the network."
https://sites.google.com/view/creditassignmentindlanddrl/home#h.p_R-9af1GhoCjr
https://beyondbackprop.github.io/
I wish I could find the Beyond Backprop talk by Schmidhuber. Instead I only found this site.
https://arxiv.org/abs/2012.14905 Meta Learning Backpropagation And Improving It
Many concepts have been proposed for meta learning with neural networks (NNs), e.g., NNs that learn to control fast weights, hyper networks, learned learning rules, and meta recurrent NNs. Our Variable Shared Meta Learning (VS-ML) unifies the above and demonstrates that simple weight-sharing and sparsity in an NN is sufficient to express powerful learning algorithms (LAs) in a reusable fashion. A simple implementation of VS-ML called VS-ML RNN allows for implementing the backpropagation LA solely by running an RNN in forward-mode. It can even meta-learn new LAs that improve upon backpropagation and generalize to datasets outside of the meta training distribution without explicit gradient calculation. Introspection reveals that our meta-learned LAs learn qualitatively different from gradient descent through fast association.
This paper sounds super interesting, I'll admit.
https://people.idsia.ch/~juergen/2010s-our-decade-of-deep-learning.html
I do not feel like watching the BB vids. Let me read the paper and article by Jurgen.
5:25pm. The intro to the paper is amazing.
The shift from standard machine learning to meta learning involves learning the learning algorithm (LA) itself, reducing the burden on the human designer to craft appropriate learning algorithms (Schmidhuber, 1987). Recent meta learning has primarily focused on similar tasks such as few-shot learning (Finn et al., 2017) or adaptation to related environments or goals (Houthooft et al., 2018). This is in stark contrast to human-engineered LAs that generalize across a wide range of datasets or environments. Very recently, it was demonstrated that meta learning can also successfully generate LAs that generalize across wide spectra of environments (Kirsch et al., 2020; Alet et al., 2020; Oh et al., 2020), e.g., from toy environments to Mujoco and Atari. Unfortunately, however, the large number of human-designed and unmodifiable inner-loop components (e.g., backpropagation) in many recent algorithms remains a limitation.
Is it possible to implement modifiable versions of backpropagation or related algorithms as part of the end-to-end differentiable activation dynamics of a neural net (NN), instead of inserting them as separate fixed routines? Here we propose the Variable Shared Meta Learning (VS-ML) principle for this purpose. It introduces a novel way of using sparsity and weight-sharing in NNs for meta learning. We build on the arguably simplest neural meta learner, the meta recurrent neural network (Meta RNN) (Hochreiter et al., 2001; Duan et al., 2016; Wang et al., 2016). We then show that VS-ML can implement certain end-to-end differentiable fast weights (Schmidhuber, 1992; 1993a; Ba et al., 2016; Schlag & Schmidhuber, 2017), hyper networks (Ha et al., 2016)), learned learning rules (Bengio et al., 1992; Gregor, 2020; Randazzo et al., 2020), or hebbian-like synaptic plasticity (Schmidhuber, 1991; 1993a; Miconi et al., 2018; 2019; Najarro & Risi, 2020) as special cases. Our mechanism, VS-ML, can implement backpropagation solely in the forward-dynamics of an RNN. Consequently, it enables meta-optimization of backprop-like algorithms. In fact, our system meta learns LAs from scratch that learn faster than online backpropagation and generalize to datasets outside of the meta training distribution. Our VS-ML RNN is the first neural meta learner without hard-coded backpropagationthat shows such strong generalization. Our results may also be of interest with regard to biologically plausible versions of backpropagation.
VS-ML blurs the semantic distinction between activations, weights, and LAs. We show how these concepts are emergent in ‘neural’ networks with ‘weight’ sharing. VS-ML learns the function each neuron performs and its LA.
I am making a strange face as I read this.
I collated my experiences and made that training scheme, but now that I've reached my limit I still find myself rattling in a slightly bigger box than a few years ago. These guys want to go far beyond.
If Jurgen or anybody else can help me exceed my limits, I do not mind heaping praise upon praise on them and considering them my superior. I know better than anybody that my performance has not been much. I cannot say that I did not put in the effort either. After six years, what I have now is exactly how good I am. I cannot say to myself that if I tried, I could do it.
5:55pm. There isn't anything I can use in the paper. They use backprop to train the metalearner too. Let me read the article.
6:05pm. Done. There were more references than actual text.
https://arxiv.org/abs/1608.05343 Decoupled Neural Interfaces using Synthetic Gradients
I won't bother reading this. This stuff is 5 years old. Let me take a look at the Beyond Backprop vids.
https://slideslive.com/38938066/bioconstrained-intelligence-and-the-great-filter
Let me give this a shot.
“Modern” backpropagation (Linnainmaa, 1970) is now widely used. Many tasks, however, cannot be solved by backprop. I give examples where credit assignment can be achieved or greatly improved through other methods such as artificial evolution, compressed network search, universal search, the Optimal Ordered Problem Solver, meta-learning.
I really wish I could find this talk by Schmidhuber.
6:20pm. https://slideslive.com/38938066/bioconstrained-intelligence-and-the-great-filter
The first talk is not that bad. Surprisingly greedy infomax (GIM) outperforms supervised end to end learning. Maybe in the future backprop's use will be to just do local optimization.
I did a great deal to infer all those tricks to make it work better, but large and non interlaping timescales across modules are hell to deal with for me.
https://slideslive.com/38938068/gradientfree-learning-with-nevergrad
Let me check out these non-gradient optimization things.
7pm. Done with lunch. Let me continue for longer.
https://slideslive.com/38942370/randomized-automatic-differentiation
Let me watch this.
7:15pm. https://slideslive.com/38942061/zorb-a-derivativefree-backpropagation-algorithm-for-neural-networks
Let me go for this. The previous video is a snoozefest.
Actually, this is interesting. It is similar to the reverse learning idea that I had.
https://arxiv.org/abs/2011.08895 ZORB - A Derivative-Free Backpropagation Algorithm for Neural Networks
I am actually pretty happy when I see somebody doing what I thought of in the past. No way can I take the time to do these experiments.
7:45pm. https://slideslive.com/38938065/hypernets-and-the-inverse-function-theorem
Let me watch this. ZORB is actually quite different what I had in mind for reversible learning in terms of implementation. I do not understand the scale and shift operations. But I guess it is fine.
I am not going to be diving into this. Let me watch this talk as well.
8:10pm. Had to take a break. Let me resume.
ZORB really caught my attention. I'll link it along with GMN paper in the next PL update. It is remarkable how faster the training is.
But though it caught my attention, I am not interested in it. Right now I am looking for an answer to the question of long term credit assignment. So ZORB is besides the point here, it has the same weaknesses as backprop in that regard.
8:30pm. https://slideslive.com/38938070/new-old-tricks-for-online-metalearning
This is the last one. Let me just go for 10m and then I am finally done for the day.
8:40pm. Nothing interesting there. For a workshop titled Beyond Backprop, there wasn't much on going beyond backprop there. ZORB and GIM lived up to the title, but the rest did not.
https://arxiv.org/abs/2008.01342 LoCo: Local Contrastive Representation Learning
https://arxiv.org/abs/2101.03419 Training Deep Architectures Without End-to-End Backpropagation: A Brief Survey
I am going to die if I try going for this right now so let me stop it here.
8:45pm. There is no point to doing this research. I am just to anxius about this issue to give it a rest properly. I am going to look into synthetic gradient methods, GIM and what else is related to them.
Once I get this out of my system, maybe at that point I will be able to start doing some real programming. I am almost at the threeshold, but need to make that final step."
Well there's a lot that happend. First off, don't look at the Contact section; I like sort of started it and it's a hot mess. But I changed the color of the header and added a bar of the same color between work and about. I also put the header for About in the middle and added a text shadow to it. And now Work, oh boy, work. I spent a great deal of time pixel pushing this section, so forgive me if I can't remember everything I changed about it. The big things arethat I set its height and then snapped it to a new height at 480px. This made it so the size of the tv monitor stays the same size and position, making landing the display behind it and the links on top of it much easier. I also moved the Work title to the middle, made it big, and gave it a glowing text-shadow. I also think I added the glow behind the cards on hover this commit. It might have been last commit. Sorry. Changed some semantics in the css to make the play icon stay centered better. I also changed the background of the whole site to black to give a more cinematic feel to the about section when the background starts showing out the sides when the screen is so big (that was a runon sentence if I ever saw one). I honestly spent so much time on this commit and tried (and failed) at so many little touches that got thrown out that it's hard to remember what I did. I need to start taking smaller bites.
Hoplomachus Champion, New Abilities, and Bug Fixes
A new champion, Hoplomachus, has been added to the game! Currently, because of his superior number of abilities, is considered the superior champion in the game.
This also means new abilities to the game. Hoplite Tradition: An exclusive passive ability that allows the user to have a 50% chance to heal for 20 when another champion is dealt a Queen or higher. (This excludes the starting hand and club trading.) Hoplite Shield: An exclusive defense bonus that allows the user to have a 20% chance to cut incoming damage by half. This chance increases to 50% if the damage is lethal.
This update also addresses numerous bugs and loops that were caused by the migration of a universal card playing system, but I was too stupid and ignorant to find on the first update. This means that this is the first update to be fully playable since commits from a week ago ;(. Apart from the biggest bugs, some values have also been tweaked to make the bot more aggressive and prone to attacking, as I found in beta testing that the bots rarely attacked the player and/or made risks that might've saved/cost their game. Hopefully tweaking these values make the game more dynamic.
In the next upcoming commits, I am going to focus on creating multiple difficulty modes, so that beginners can play the game without trouble whilst veterans (like me) can still destroy the hardest difficulty. I would also like to make a tutorial scene soon, but that's probably going to take a while, so stay tuned for that.
feat(Holy fucking shit 155 changes)
Painting puzzle is now done Timer now works with UI and explosion Opening Dialogue is implemented