Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IlegalAccessError throws when using browsermob with WebDriver 2.30.0 to capture n/w #90

Open
JaneLee opened this issue Mar 13, 2013 · 29 comments

Comments

@JaneLee
Copy link

JaneLee commented Mar 13, 2013

Hi , lightbody,

I was so excited after reading a blog about how to capture n/w using WebDriver. One solution is to use browsermob-proxy. But unfortunately I met one issue during my practise project with latest firefox version. And it works not OK etither on IE8 or on Chrome browsers. Causing we need to cover mutliple browsers testing. Pls let me know how to solve this issue! Thx


Running environment:

Firefox 19.0.2
WebDriver-Server >= 2.27.0


Maven dependency configurations:

<dependencies>
    <dependency>
        <groupId>commons-lang</groupId>
        <artifactId>commons-lang</artifactId>
        <version>2.6</version>
    </dependency>
    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.8</version>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-server</artifactId>
        <version>2.30.0</version>
    </dependency>
    <dependency>
        <groupId>biz.neustar</groupId>
        <artifactId>browsermob-proxy</artifactId>
        <version>2.0-beta-7</version>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.seleniumhq.selenium</groupId>
                <artifactId>selenium-api</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

</dependencies>

My code using browsermob to capture n/w

@Test
// TODO: using Firefox failed
public void testFirefoxWithOBMP() throws Exception {
    // start the proxy
    ProxyServer server = new ProxyServer(5555);
    server.start();

    // get the Selenium proxy object
    Proxy proxy = server.seleniumProxy();

    // configure it as a desired capability
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability(CapabilityType.PROXY, proxy);

    // start the browser up
    System.setProperty("webdriver.firefox.bin", "D:\\Program Files\\Mozilla Firefox\\firefox.exe");
    WebDriver driver = new FirefoxDriver(capabilities);

    // create a new HAR with the label "innerhotsale"
    server.newHar("innerhotsale");

    // open re.taobao.com/search?
    driver.get(INNER_HOTSALE_URL);
    driver.manage().window().maximize();

    // get the HAR data
    Har har = server.getHar();

    System.out.println("============" + har.getLog().getEntries().size());

    System.out.println("END!!");

    server.stop();
}

Following are error logs:

INFO 03/13 07:12:09 o.b.p.j.h.HttpServer - Version Jetty/5.1.x
INFO 03/13 07:12:09 o.b.p.j.u.Container - Started HttpContext[/,/]
INFO 03/13 07:12:09 o.b.p.j.h.SocketLis~ - Started SocketListener on 0.0.0.0:5555
INFO 03/13 07:12:09 o.b.p.j.u.Container - Started org.browsermob.proxy.jetty.jetty.Server@12be1bd
WARN 03/13 07:12:23 o.b.p.j.h.HttpConne~ - GET http://re.taobao.com/search? HTTP/1.1
java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory
at org.browsermob.proxy.http.SimulatedSocketFactory.connectSocket(SimulatedSocketFactory.java:118)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:561)
at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:372)
at org.browsermob.proxy.http.BrowserMobHttpRequest.execute(BrowserMobHttpRequest.java:157)
at org.browsermob.proxy.BrowserMobProxyHandler.proxyPlainTextRequest(BrowserMobProxyHandler.java:270)
at org.browsermob.proxy.selenium.SeleniumProxyHandler.handle(SeleniumProxyHandler.java:185)
at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1509)
at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1461)
at org.browsermob.proxy.jetty.http.HttpServer.service(HttpServer.java:892)
at org.browsermob.proxy.jetty.http.HttpConnection.service(HttpConnection.java:815)
at org.browsermob.proxy.jetty.http.HttpConnection.handleNext(HttpConnection.java:981)
at org.browsermob.proxy.jetty.http.HttpConnection.handle(HttpConnection.java:832)
at org.browsermob.proxy.jetty.http.SocketListener.handleConnection(SocketListener.java:245)
at org.browsermob.proxy.jetty.util.ThreadedServer.handle(ThreadedServer.java:357)
at org.browsermob.proxy.jetty.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
WARN 03/13 07:12:24 o.b.p.j.h.HttpConne~ - GET http://re.taobao.com/favicon.ico HTTP/1.1
java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory
at org.browsermob.proxy.http.SimulatedSocketFactory.connectSocket(SimulatedSocketFactory.java:118)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:561)
at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:372)
at org.browsermob.proxy.http.BrowserMobHttpRequest.execute(BrowserMobHttpRequest.java:157)
at org.browsermob.proxy.BrowserMobProxyHandler.proxyPlainTextRequest(BrowserMobProxyHandler.java:270)
at org.browsermob.proxy.selenium.SeleniumProxyHandler.handle(SeleniumProxyHandler.java:185)
at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1509)
at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1461)
at org.browsermob.proxy.jetty.http.HttpServer.service(HttpServer.java:892)
at org.browsermob.proxy.jetty.http.HttpConnection.service(HttpConnection.java:815)
at org.browsermob.proxy.jetty.http.HttpConnection.handleNext(HttpConnection.java:981)
at org.browsermob.proxy.jetty.http.HttpConnection.handle(HttpConnection.java:832)
at org.browsermob.proxy.jetty.http.SocketListener.handleConnection(SocketListener.java:245)
at org.browsermob.proxy.jetty.util.ThreadedServer.handle(ThreadedServer.java:357)
at org.browsermob.proxy.jetty.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
============2
END!!
INFO 03/13 07:12:24 o.b.p.j.u.ThreadedS~ - Stopping Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=5555]
INFO 03/13 07:12:24 o.b.p.j.h.SocketLis~ - Stopped SocketListener on 0.0.0.0:5555
java.lang.NullPointerException
at com.etao.ad.p4p.hotsale.learn.testFirefox.afterClass(testFirefox.java:101)

YOURS,
Jane Li

@lightbody
Copy link
Member

Jane,
I believe the problem is that you're using an old version of Java. We require Java 1.6 and it looks like you're running 1.5. Can you confirm?

@JaneLee
Copy link
Author

JaneLee commented Mar 14, 2013

Hi, lightbody,

My java version is "1.6.0_07".

The browsermob dependency of selenium-api version is 2.20.0. We tested when my project configured with following selenium-server version, browsermob works fine. But not OK with the version larger than 2.26.0. Pls help to confirm whether the error is related to it?

YOURS,
Jane

@lightbody
Copy link
Member

Interesting. Do you have any unique policies/access rights on the JVM? How are you launching it?

java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory

On Mar 14, 2013, at 6:21 AM, Jane Li notifications@github.com wrote:

Hi, lightbody,

My java version is "1.6.0_07".

The browsermob dependency of selenium-api version is 2.20.0. We tested when my project configured with following selenium-server version, browsermob works fine. But not OK on the version larger than 2.26.0. pls help to confirm whether the error is related to it?

org.seleniumhq.selenium
selenium-server
2.20.0

YOURS,
Jane


Reply to this email directly or view it on GitHub.

@schwartzie
Copy link

I'm getting the exact same error using the 2.0-beta-7 standalone REST API. I'm using Java version 1.6.0_26 on a 64-bit Debian Squeeze VM.

@lightbody
Copy link
Member

Can you boil it down to a bare bones test case?

On Mar 14, 2013, at 10:23 AM, schwartzie notifications@github.com wrote:

I'm getting the exact same error using the 2.0-beta-7 standalone REST API. I'm using Java version 1.6.0_26 on a 64-bit Debian Squeeze VM.


Reply to this email directly or view it on GitHub.

@schwartzie
Copy link

I didn't get any farther than a bare bones test case myself, so here's a complete rundown of what I did:

Launch proxy server:

$ cd browsermob-proxy-2.0-beta-7/bin/
$ sh browsermob-proxy -port 9090
INFO 03/14 18:14:12 o.b.p.Main           - Starting BrowserMob Proxy version 2.0-beta-7
INFO 03/14 18:14:12 o.e.j.u.log          - jetty-7.3.0.v20110203
INFO 03/14 18:14:12 o.e.j.u.log          - started o.e.j.s.ServletContextHandler{/,null}
INFO 03/14 18:14:12 o.e.j.u.log          - Started SelectChannelConnector@0.0.0.0:9090

Create proxy instance, and add page to HAR:

$ curl -X POST http://localhost:9090/proxy
{"port":9091}
$ curl -X PUT http://localhost:9090/proxy/9091/har

...results in the following output from the server:

INFO 03/14 18:14:31 o.b.p.j.h.HttpServer - Version Jetty/5.1.x
INFO 03/14 18:14:31 o.b.p.j.u.Container  - Started HttpContext[/,/]
INFO 03/14 18:14:31 o.b.p.j.h.SocketLis~ - Started SocketListener on 0.0.0.0:9091
INFO 03/14 18:14:31 o.b.p.j.u.Container  - Started org.browsermob.proxy.jetty.jetty.Server@785e8d7d

Make a request via the proxy:

$ curl -iL -x localhost:9091 http://google.com
HTTP/1.1 500 Internal Server Error
Date: Thu, 14 Mar 2013 18:14:58 GMT
Server: Jetty/5.1.x (Linux/2.6.32-5-xen-amd64 amd64 java/1.6.0_26
Connection: close

...results in the following output from the server:

WARN 03/14 18:14:59 o.b.p.j.h.HttpConne~ - GET http://google.com/ HTTP/1.1
java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory
    at org.browsermob.proxy.http.SimulatedSocketFactory.connectSocket(SimulatedSocketFactory.java:118)
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
    at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
    at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
    at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:561)
    at org.browsermob.proxy.http.BrowserMobHttpClient.execute(BrowserMobHttpClient.java:372)
    at org.browsermob.proxy.http.BrowserMobHttpRequest.execute(BrowserMobHttpRequest.java:157)
    at org.browsermob.proxy.BrowserMobProxyHandler.proxyPlainTextRequest(BrowserMobProxyHandler.java:270)
    at org.browsermob.proxy.selenium.SeleniumProxyHandler.handle(SeleniumProxyHandler.java:185)
    at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1509)
    at org.browsermob.proxy.jetty.http.HttpContext.handle(HttpContext.java:1461)
    at org.browsermob.proxy.jetty.http.HttpServer.service(HttpServer.java:892)
    at org.browsermob.proxy.jetty.http.HttpConnection.service(HttpConnection.java:815)
    at org.browsermob.proxy.jetty.http.HttpConnection.handleNext(HttpConnection.java:981)
    at org.browsermob.proxy.jetty.http.HttpConnection.handle(HttpConnection.java:832)
    at org.browsermob.proxy.jetty.http.SocketListener.handleConnection(SocketListener.java:245)
    at org.browsermob.proxy.jetty.util.ThreadedServer.handle(ThreadedServer.java:357)
    at org.browsermob.proxy.jetty.util.ThreadPool$PoolThread.run(ThreadPool.java:534)

Retrieve HAR:

$ curl http://localhost:9090/proxy/9091/har
{"log":{"entries":[{"time":0,"request":{"method":"GET","headers":[],"queryString":[],"cookies":[],"bodySize":0,"url":"http://google.com/","httpVersion":"HTTP","headersSize":0},"response":{"content":{"size":0,"mimeType":""},"headers":[],"status":-999,"cookies":[],"bodySize":0,"httpVersion":"HTTP","headersSize":0,"statusText":"NO RESPONSE","redirectURL":""},"startedDateTime":"2013-03-14T18:14:58.926+0000","pageref":"Page 1","cache":{}}],"version":"1.1","browser":{"name":"cURL","version":"7.21.0"},"creator":{"name":"BrowserMob Proxy","version":"2.0"},"pages":[{"id":"Page 1","pageTimings":{},"startedDateTime":"2013-03-14T18:14:44.241+0000","title":""}]}}

Let me know if there's anything else I can provide that will help.

@JaneLee
Copy link
Author

JaneLee commented Mar 16, 2013

HI, lightbody,

To getting more clear with this issue, we tried to test on much more different scenarios.

Those are our testing result:

  1. webdriver-server's version 2.30.0 vs browsermob-proxy'version 2.0-beta-6 =====> works fine
  2. webdriver-server's version 2.30.0 vs browsermob-proxy'version 2.0-beta-7 =====> throws IlegalAccessError always.

Maybe this result can do some help. Pls help to check.

YOURS,
Jane

@schwartzie
Copy link

I just tried my test case with 2.0-beta-6, and it worked for me too.

@JaneLee
Copy link
Author

JaneLee commented Mar 18, 2013

will this issue be resolved at 2.0-beta-7 version?

@lightbody
Copy link
Member

OK it looks like this all came from this comment:

e4c53e1#src/main/java/org/browsermob/proxy/http/SimulatedSocketFactory.java

I'm hoping that @rexhoffman can help us get it resolved. Rex?

@rexhoffman
Copy link
Contributor

Hmm... I was using a mac os x version of java 6 when I wrote this, and didn't experience any security based exceptions/issues. I started a new gig since writing this however and don't have any tests set up around the code.

What os/jvms were used?

I have no need to support java 6 at this point. Just seemed a little sad to force 1.7 for one method. I believe the there were concerns regarding the time constraints of using other methods to determine the host. Though we never discussed caching.

Having done some work configuring policies/building hacks to prevent resource access during testing, this smells like a jvm doing security policy checking in the method.

Java security models are painful at best, and seems to diverge between different jvm providers, so my hack to work around getting access to non-public methods should probably be reverted -- since it has been proven to break with some jvms default policy configurations.

I'm not sure if another work-around would be accepted targeting some other method to calculate the host and potentially cache it.

Lesson learned: don't use reflection to grab non-public methods in the jvm -- they may get tightened up/be tighter in another jvm.

@lightbody
Copy link
Member

@JaneLee and @schwartzie - can you get back to @rexhoffman with any info you have about the issue?

@JaneLee
Copy link
Author

JaneLee commented Mar 27, 2013

@rexhoffman @lightbody

My OS is window-xp; And using default jvm configration in Intellij IDEA.

@akosicki
Copy link

akosicki commented Apr 3, 2013

Just confirming the bug, java.lang.IllegalAccessError is being thrown when trying to use proxy.

Environment:

# browsermob-proxy version: 2.0-beta-7
$ uname -a
Linux aleksander-cuda 3.2.0-39-generic-pae #62-Ubuntu SMP Wed Feb 27 22:25:11 UTC 2013 i686 i686 i386 GNU/Linux
$ java -version
java version "1.6.0_43"
Java(TM) SE Runtime Environment (build 1.6.0_43-b01)
Java HotSpot(TM) Server VM (build 20.14-b01, mixed mode)

Steps to reproduce:

# as in tutotorial
$ sh browsermob-proxy -port 9090 &
...
$ curl -X POST http://localhost:9090/proxy
{"port":9091}
$ curl -x localhost:9091 http://google.com
java.lang.IllegalAccessError: tried to access method java.net.InetSocketAddress.getHostString()Ljava/lang/String; from class org.browsermob.proxy.http.SimulatedSocketFactory
    at org.browsermob.proxy.http.SimulatedSocketFactory.connectSocket(SimulatedSocketFactory.java:118)
...

@hyeung-trulia
Copy link

I got the same error on mac with JDK 1.7.

@lightbody
Copy link
Member

@rexhoffman What do you suggest we do? Looks like quite a few people are having trouble and I'd like to get this resolved before making the next release.

@rexhoffman
Copy link
Contributor

Definitely surprised by the jdk7 issue on mac as the method was supposed to
be publicly accessible in java 7. A security manager could cause it
however (limiting access to network method)

As per #74

My understand is that we wanted to use that method for performance reasons
(to avoid a reverse dns lookup). There was discussion to drop support for
java 6 because of the method being package scoped.

We don't have many good options. Tightening java security/desire for a
fast method/Desire to support java 6 all seem to conflict.

Personally I'd think taking the performance hit and using

http://docs.oracle.com/javase/6/docs/api/java/net/InetSocketAddress.html#getHostName()is
the right way to go.

wdyt?

I'd have no problem doing this tomorrow and putting this issue down. I'll
also take another look at the code and write a test for this, but I had
used the code without issue after writing. So not expecting to find a
simple bug.

On Sun, Apr 7, 2013 at 2:29 PM, Patrick Lightbody
notifications@github.comwrote:

@rexhoffman https://github.com/rexhoffman What do you suggest we do?
Looks like quite a few people are having trouble and I'd like to get this
resolved before making the next release.


Reply to this email directly or view it on GitHubhttps://github.com//issues/90#issuecomment-16024666
.

@annagrgr
Copy link

Hi, I'm getting this error too, 2.0-beta-7, java 6.

@lightbody
Copy link
Member

I will get this fixed soon.

On Fri, Apr 19, 2013 at 5:00 AM, annagrgr notifications@github.com wrote:

Hi, I'm getting this error too, 2.0-beta-7, java 6.


Reply to this email directly or view it on GitHubhttps://github.com//issues/90#issuecomment-16649304
.

@cpuser
Copy link

cpuser commented Apr 22, 2013

can confirm this error: MacOS X, java version "1.6.0_43", 2.0-beta-7

@lightbody
Copy link
Member

I've got it reproduced. Now to fix it. Should be tonight I hope. 

Sent from Mailbox for iPhone

On Mon, Apr 22, 2013 at 3:08 AM, cpuser notifications@github.com wrote:

can confirm this error: MacOS X, java version "1.6.0_43", 2.0-beta-7

Reply to this email directly or view it on GitHub:
#90 (comment)

@davehunt
Copy link
Contributor

I'm also seeing this. Hoping for a beta-8 soon! :)

@lightbody
Copy link
Member

I just committed a fix for this. For those of you who can build from master, please try it out and confirm it works for you now. I hope to release beta-8 soon!

@davehunt
Copy link
Contributor

I'm getting failures when attempting to build from master. See http://www.pastebin.mozilla.org/2359519 for details. If you're able to build, could you make a release candidate available? I'm happy to test it.

@lightbody
Copy link
Member

@davehunt That was actually a goof on my part. Try updating again and doing the build. Everyone else is encouraged to try this latest build out too, since to fully resolve this bug I upgraded from HttpClient 4.1 to 4.2 and it might introduce some weird artifacts.

@davehunt
Copy link
Contributor

@lightbody beta 8 works for me! :)

@breinjhel
Copy link

Hi,

Where can I find beta 8? Im using maven dependency by the way.

Thanks.

@breinjhel
Copy link

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants