Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bluecoat Categorization failes #216

Closed
crackytsi opened this issue Apr 3, 2018 · 5 comments
Closed

Bluecoat Categorization failes #216

crackytsi opened this issue Apr 3, 2018 · 5 comments
Labels
category:bug Issue is related to a bug scope:analyzer Issue is analyzer related
Milestone
1.9.3

Comments

@crackytsi
Copy link
Contributor

crackytsi commented Apr 3, 2018 •
edited
Loading

Request Type

Bug

Work Environment

Question Answer
OS version (server) Debian, Ubuntu, CentOS, RedHat, ...
OS version (client) Seven
Cortex Analyzer Name BlueCoat_Categorization_1_0
Cortex Analyzer Version 1.9.1 (latest from today)
Cortex Version 2.0.1

Description

BlueCoat analyzers failes

Steps to Reproduce

  1. Install Cortex2
  2. Install latest CortexAnalyzers
  3. Run BlueCoat Analysis with Cortex2 for domain: www.google.de

Complementary information

Invalid output
Traceback (most recent call last):
  File "Bluecoat/categorization.py", line 53, in call_bluecoat_api
    return json.loads(BC_json_answer.text)
  File "/usr/lib/python3.4/json/__init__.py", line 318, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.4/json/decoder.py", line 343, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.4/json/decoder.py", line 361, in raw_decode
    raise ValueError(errmsg("Expecting value", s, err.value)) from None
ValueError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Bluecoat/categorization.py", line 108, in <module>
    BluecoatAnalyzer().run()
  File "Bluecoat/categorization.py", line 91, in run
    json_answer = self.call_bluecoat_api(self.get_data())
  File "Bluecoat/categorization.py", line 55, in call_bluecoat_api
    self.error(e)
  File "/usr/local/lib/python3.4/dist-packages/cortexutils/analyzer.py", line 182, in error
    ensure_ascii=ensure_ascii)
  File "/usr/lib/python3.4/json/__init__.py", line 178, in dump
    for chunk in iterable:
  File "/usr/lib/python3.4/json/encoder.py", line 422, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/lib/python3.4/json/encoder.py", line 396, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.4/json/encoder.py", line 429, in _iterencode
    o = _default(o)
  File "/usr/lib/python3.4/json/encoder.py", line 173, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: ValueError('Expecting value: line 1 column 1 (char 0)',) is not JSON serializable
{"errorMessage":
@nadouani nadouani added this to the 1.9.2 milestone Apr 3, 2018
@nadouani nadouani self-assigned this Apr 3, 2018
@nadouani
Copy link
Contributor

nadouani commented Apr 3, 2018

Well looks like Bluecoat is returning something like:

<html>

<head>
  <title>Apache Tomcat/7.0.52 (Ubuntu) - Error report</title>
  <style>
    ...
  </style>
</head>

<body>
  <h1>HTTP Status 403 - Security violation.</h1>
  <HR size="1" noshade="noshade">
  <p><b>type</b> Status report</p>
  <p><b>message</b> <u>Security violation.</u></p>
  <p><b>description</b> <u>Access to the specified resource has been forbidden.</u></p>
  <HR size="1" noshade="noshade">
  <h3>Apache Tomcat/7.0.52 (Ubuntu)</h3></body>

</html>

which is not what the analyzer expects.

Two questions:

  • Is the API used by the analyzer still valid?
  • Should the analyzer handle the 403 error?

@nadouani
Copy link
Contributor

nadouani commented Apr 3, 2018

BTW, This is not related to Cortex 2.0.0

@nadouani nadouani added category:bug Issue is related to a bug scope:analyzer Issue is analyzer related and removed scope:cortex labels Apr 3, 2018
@nadouani nadouani removed this from the 1.9.2 milestone Apr 3, 2018
@crackytsi crackytsi changed the title Bluecoat Categorization failes with Cortex2 Bluecoat Categorization failes Apr 3, 2018
@3c7
Copy link
Contributor

3c7 commented Apr 3, 2018

Symantec added Javascript to the site that prevents the analyzer from working. Querying Bluecoat using curl in the previous successful manner is currently not possible, too. Bad timing. Need to check, if we can fix it.

@saadkadhi
Copy link
Contributor

If we can't fix it, we need to ax it.

@3c7 3c7 added this to the 1.9.3 milestone Apr 6, 2018
@nadouani nadouani removed their assignment Apr 6, 2018
@3c7
Copy link
Contributor

3c7 commented Apr 6, 2018

Actually, querying Symantecs Sitereview is against their ToS. So, for now, I'll remove the analyzer. Fortinet can be used for website categorization anyway.

3c7 added a commit that referenced this issue Apr 6, 2018
@3c7
Fixes #216 (not rly): removed Bluecoat analyzer due to incompatibilit…
2cfdf6b 
…y and ToS violation.
@nadouani nadouani closed this as completed Apr 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
category:bug Issue is related to a bug scope:analyzer Issue is analyzer related
Projects
None yet
Milestone
1.9.3
Development

No branches or pull requests
4 participants
@nadouani @3c7 @saadkadhi @crackytsi