stale Broadcaster issue

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

stale Broadcaster issue

jurjenvg
Hi all,

I'm using GF 3.1 native comet with Atmosphere 0.7.2. Am trying to remove a broadcaster which is no longer used as follows:

     Broadcaster oldBroadcaster = resource.getBroadcaster();
     oldBroadcaster.removeAtmosphereResource(resource.getAtmosphereResource());
     BroadcasterFactory.getDefault().remove(oldBroadcaster, oldBroadcaster.getID());
     oldBroadcaster.destroy();

When iterating over all broadcasters in an EJB singleton per below method, the above broadcaster is still found (multiple times).


    @Schedule(second = "*", minute = "*", hour = "*", info="every second")
    private void broadCast()
    {
        for (Broadcaster b : BroadcasterFactory.getDefault().lookupAll() )
        {
            final String message = AtmosphereBroadcasterBean.class.getSimpleName() + ": hello " + b.getID() + "[" + messages++ + "]";
           
            b.broadcast(message);
           
            log.info(b.getID() + "->" + message);
        }
    }
The result is an Exception with below root cause/stacktrace:

SEVERE: 116750 [Grizzly-kernel-thread(1)] WARN org.atmosphere.cpr.DefaultBroadcasterFactory - This Broadcaster has been destroyed and cannot be used

SEVERE: java.lang.IllegalStateException: This Broadcaster has been destroyed and cannot be used
    at org.atmosphere.cpr.DefaultBroadcaster.removeAtmosphereResource(DefaultBroadcaster.java:741)
    at org.atmosphere.cpr.DefaultBroadcasterFactory.removeAllAtmosphereResource(DefaultBroadcasterFactory.java:204)
    at org.atmosphere.cpr.AsynchronousProcessor.destroyResource(AsynchronousProcessor.java:377)
    at org.atmosphere.cpr.AsynchronousProcessor.cancelled(AsynchronousProcessor.java:424)
    at org.atmosphere.container.GrizzlyCometSupport.cancelled(GrizzlyCometSupport.java:180)
    at org.atmosphere.container.GrizzlyCometSupport$VoidCometHandler.onInterrupt(GrizzlyCometSupport.java:235)
    at com.sun.grizzly.comet.CometEngine.interrupt0(CometEngine.java:388)
    at com.sun.grizzly.comet.CometTask.run(CometTask.java:113)
    at com.sun.grizzly.comet.CometEngine.interrupt(CometEngine.java:369)
    at com.sun.grizzly.comet.CometTask.checkIfClientClosedConnection(CometTask.java:190)
    at com.sun.grizzly.comet.CometTask.handleSelectedKey(CometTask.java:171)
    at com.sun.grizzly.SelectorHandlerRunner.handleSelectedKey(SelectorHandlerRunner.java:279)
    at com.sun.grizzly.SelectorHandlerRunner.handleSelectedKeys(SelectorHandlerRunner.java:263)
    at com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:200)
    at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:132)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)

My questions are:
  1. How to remove a broadcaster such that the framework completely removes all references to it, in a threadsafe way?
  2. If above is impossible: how to detect that a broadcaster has been destroyed?
  3. How to detect that a client has disconnected?
Thanks,

Jurjen
Reply | Threaded
Open this post in threaded view
|

Re: stale Broadcaster issue

Jeanfrancois Arcand-4
Salut,

On 11-06-20 4:41 AM, Jurjen van Geenen wrote:

> Hi all,
>
> I'm using GF 3.1 native comet with Atmosphere 0.7.2. Am trying to remove
> a broadcaster which is no longer used as follows:
>
>       Broadcaster oldBroadcaster = resource.getBroadcaster();
>
> oldBroadcaster.removeAtmosphereResource(resource.getAtmosphereResource());
>       BroadcasterFactory.getDefault().remove(oldBroadcaster,
> oldBroadcaster.getID());
>       oldBroadcaster.destroy();
>
> When iterating over all broadcasters in an EJB singleton per below
> method, the above broadcaster is still found (multiple times).
>
>
>      @Schedule(second = "*", minute = "*", hour = "*", info="every second")
>      private void broadCast()
>      {
>          for (Broadcaster b : BroadcasterFactory.getDefault().lookupAll() )
>          {
>              final String message =
> AtmosphereBroadcasterBean.class.getSimpleName() + ": hello " + b.getID()
> + "[" + messages++ + "]";
>
>              b.broadcast(message);
>
> log.info <http://log.info>(b.getID() + "->" + message);

Can you cut & paste your here? Technically the same broadcaster
shouldn't appear more than one.


>          }
>      }
> The result is an Exception with below root cause/stacktrace:
>
> SEVERE: 116750 [Grizzly-kernel-thread(1)] WARN
> org.atmosphere.cpr.DefaultBroadcasterFactory - This Broadcaster has been
> destroyed and cannot be used
>
> SEVERE: java.lang.IllegalStateException: This Broadcaster has been
> destroyed and cannot be used
>      at
> org.atmosphere.cpr.DefaultBroadcaster.removeAtmosphereResource(DefaultBroadcaster.java:741)
>      at
> org.atmosphere.cpr.DefaultBroadcasterFactory.removeAllAtmosphereResource(DefaultBroadcasterFactory.java:204)
>      at
> org.atmosphere.cpr.AsynchronousProcessor.destroyResource(AsynchronousProcessor.java:377)
>      at
> org.atmosphere.cpr.AsynchronousProcessor.cancelled(AsynchronousProcessor.java:424)
>      at
> org.atmosphere.container.GrizzlyCometSupport.cancelled(GrizzlyCometSupport.java:180)
>      at
> org.atmosphere.container.GrizzlyCometSupport$VoidCometHandler.onInterrupt(GrizzlyCometSupport.java:235)
>      at com.sun.grizzly.comet.CometEngine.interrupt0(CometEngine.java:388)
>      at com.sun.grizzly.comet.CometTask.run(CometTask.java:113)
>      at com.sun.grizzly.comet.CometEngine.interrupt(CometEngine.java:369)
>      at
> com.sun.grizzly.comet.CometTask.checkIfClientClosedConnection(CometTask.java:190)
>      at
> com.sun.grizzly.comet.CometTask.handleSelectedKey(CometTask.java:171)
>      at
> com.sun.grizzly.SelectorHandlerRunner.handleSelectedKey(SelectorHandlerRunner.java:279)
>      at
> com.sun.grizzly.SelectorHandlerRunner.handleSelectedKeys(SelectorHandlerRunner.java:263)
>      at
> com.sun.grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:200)
>      at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:132)
>      at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>      at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>      at java.lang.Thread.run(Thread.java:619)
>
> My questions are:
>
>    1. How to remove a broadcaster such that the framework completely
>       removes all references to it, in a threadsafe way?

OK the current exception means the client disconnected and when Grizzly
detected it, it tried to clean up the broadcaster. I will take a look as
this is a bug (e.g. the exception shouldn't occur).

You can use a BroadcasterLifeCyclePolicy in order to tell Atmosphere to
get rid of unused Broadcaster:

   http://is.gd/0o41cg

and decide which policy you want to use

   http://is.gd/MZo0dS


>    2. If above is impossible: how to detect that a broadcaster has been
>       destroyed?

This should be possible.

>    3. How to detect that a client has disconnected?

I'm guessing your are using Jersey, right? Just add an
AtmosphereResourceEventListener

   http://is.gd/pb2sHF

when you suspend the connection, and will get notified every time a
client disconnect.

A+

-- Jeanfrancois


>
> Thanks,
>
> Jurjen
sd
Reply | Threaded
Open this post in threaded view
|

Re: stale Broadcaster issue

sd
In reply to this post by jurjenvg
I'm seeing the same issue with the Redis Broadcaster after about a day

011-10-06 14:04:07,720 ["http-bio-8283"-exec-170] ERROR cpr.AtmosphereServlet  - AtmosphereServlet exception
java.lang.IllegalStateException: This Broadcaster has been destroyed and cannot be used
        at org.atmosphere.cpr.DefaultBroadcaster.removeAtmosphereResource(DefaultBroadcaster.java:741)
        at org.atmosphere.cpr.AsynchronousProcessor.destroyResource(AsynchronousProcessor.java:375)
        at org.atmosphere.cpr.AsynchronousProcessor.invokeAtmosphereHandler(AsynchronousProcessor.java:368)
        at org.atmosphere.cpr.AsynchronousProcessor.timedout(AsynchronousProcessor.java:331)
....

I don't set the policy but looking at the code it defaults to NEVER, why would this be happening by itself?

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: stale Broadcaster issue

jfarcand
Administrator
Salut,

can you share a test case? Those exceptions shoudn't stop the application working...is that the case?

A+

-- Jeanfrancois
sd
Reply | Threaded
Open this post in threaded view
|

Re: stale Broadcaster issue

sd
I can't provide a test case because it is inconsistent, usually it takes close to a day and yes the application is unresponsive and I need to perform a restart.

 
Reply | Threaded
Open this post in threaded view
|

Re: stale Broadcaster issue

jfarcand
Administrator
OK thanks...can you try the latest 0.8.0-SNAPSHOT and see if that still occurs? I need to find a way to reproduce that issue....

BTW let's move the discussion to atmosphere-framework@googlegroups.com

-- Jeanfrancois