[pull] master from kevoreilly:master #702
Open
+171,482
−57,578
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Update rdp.py for class name
Update package for rdp package
Create watchdownloads.py
The `diec` binary outputs non-JSON output in JSON mode, causing the parsing to fail. For example:
```text
[!] Heuristic scan is disabled. Use --heuristicscan to enable
{
"detects": [
{
"filetype": "PE64",
"info": "",
"offset": "0",
"parentfilepart": "Header",
"size": "3488048",
"values": [
{
"info": "",
"name": "Microsoft Linker",
"string": "Linker: Microsoft Linker(14.22.27905)",
"type": "Linker",
"version": "14.22.27905"
},
{
"info": "C++",
"name": "Microsoft Visual C/C++",
"string": "Compiler: Microsoft Visual C/C++(19.22.27905)[C++]",
"type": "Compiler",
"version": "19.22.27905"
},
{
"info": "",
"name": "Visual Studio",
"string": "Tool: Visual Studio(2019, v16.2)",
"type": "Tool",
"version": "2019, v16.2"
}
]
}
]
}
```
There is a related issue noted against the DIE project:
horsicq/Detect-It-Easy#242
Ignore non-JSON output in detect-it-easy output
Update Arkei.yar Yara Rule
…avoid stale data This is achieved by calling `scoped_session().remove()` at the end of every web request. Whilst this bug didn't appear to affect anything critical, it did manifest itself in the task status views (API and UI) by ocassionally showing stale data after a task status had been updated on the backend. Using request-local sessions is the recommended approach inside web apps as noted in the SQLAlchemy [docs](https://docs.sqlalchemy.org/en/20/orm/contextual.html#using-thread-local-scope-with-web-applications): > As discussed in the section When do I construct a Session, when do I commit it, and when do I close it?, a web application is architected around the concept of a web request, and integrating such an application with the Session usually implies that the Session will be associated with that request. As it turns out, most Python web frameworks, with notable exceptions such as the asynchronous frameworks Twisted and Tornado, use threads in a simple way, such that a particular web request is received, processed, and completed within the scope of a single worker thread. When the request ends, the worker thread is released to a pool of workers where it is available to handle another request. > > This simple correspondence of web request and thread means that to associate a Session with a thread implies it is also associated with the web request running within that thread, and vice versa, provided that the Session is created only after the web request begins and torn down just before the web request ends. So it is a common practice to use scoped_session as a quick way to integrate the Session with a web application. The sequence diagram below illustrates this flow: ``` Web Server Web Framework SQLAlchemy ORM Code -------------- -------------- ------------------------------ startup -> Web framework # Session registry is established initializes Session = scoped_session(sessionmaker()) incoming web request -> web request -> # The registry is *optionally* starts # called upon explicitly to create # a Session local to the thread and/or request Session() # the Session registry can otherwise # be used at any time, creating the # request-local Session() if not present, # or returning the existing one Session.execute(select(MyClass)) # ... Session.add(some_object) # ... # if data was modified, commit the # transaction Session.commit() web request ends -> # the registry is instructed to # remove the Session Session.remove() sends output <- outgoing web <- response ```
This fixes a bug where the process function fails to update the database when setting the task status to `reported` if the pebble process had previously processed the task. By removing the session after processing, the next task that runs in the same process will grab a new session and will be forced to pull new objects from the database. Preventing any data inconsistencies.
if its not valid UTF-8 use latin1 instead to prevent reporting failure
use chardet to avoid UnicodeDecodeError due to invalid utf-8 byte string.
use chardet to avoid UnicodeDecodeError due to invalid utf-8 byte string. + fix missing import
fix litereport failure due to invalid utf-8 byte string
Reduce SQLAlchemy session scope to avoid stale data
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )