HdpH crash with polymorphic closure instances

[cirodrig]

Parallel HdpH code fails when using polymorphic list instances of ToClosure and ForceCC. It fails when a closure environment is deserialized. I guess that the closure-generating code is choosing the wrong static index of a polymorphic function, because the failure can be changed by reordering static declarations.

The following code uses a trivial parallel loop with parMapNF to demonstrate the problem. Switching from polymorphic to monomorphic instances (by changing #if 1 to #if 0) makes the problem go away.

{-# LANGUAGE CPP, TemplateHaskell, FlexibleInstances,                           
             FlexibleContexts, ScopedTypeVariables #-}

import Control.DeepSeq
import Control.Monad
import Control.Parallel.HdpH hiding(declareStatic)
import Control.Parallel.HdpH.Strategies hiding(declareStatic)
import qualified Control.Parallel.HdpH as HdpH (declareStatic)
import qualified Control.Parallel.HdpH.Strategies as HdpH.Strategies (declareSt\
atic)
import qualified Control.Parallel.MP.MPI_ByteString as MPI
import Data.Bits
import Data.List
import Data.Monoid
import Data.Serialize

declareStatic =
  mconcat
  [ declare $(static 'parWorker)
  , HdpH.declareStatic
  , HdpH.Strategies.declareStatic
  , declare (staticToClosure :: StaticToClosure Float)
  , declare (staticForceCC :: StaticForceCC Float)

  -- For a crash, these declarations must come first                            
  , declare (staticToClosure :: StaticToClosure [(Float, Float)])
  , declare (staticForceCC :: StaticForceCC [(Float, Float)])

  -- and these must come afterward                                              
  , declare (staticToClosure :: StaticToClosure [Float])
  , declare (staticForceCC :: StaticForceCC [Float])
  ]

instance ToClosure Float where locToClosure = $(here)
instance ForceCC Float where locForceCC = $(here)

instance ToClosure (Float, Float) where locToClosure = $(here)
instance ForceCC (Float, Float) where locForceCC = $(here)

#if 1
-- Causes a crash                                                               
instance (ToClosure a, NFData a) => ToClosure [a] where locToClosure = $(here)
instance (ToClosure a, NFData a) => ForceCC [a] where locForceCC = $(here)

#else
-- Doesn't crash                                                                
instance ToClosure [(Float, Float)] where locToClosure = $(here)
instance ForceCC [(Float, Float)] where locForceCC = $(here)

instance ToClosure [Float] where locToClosure = $(here)
 instance ForceCC [Float] where locForceCC = $(here)
#endif

parWorker :: Closure Float -> [Float] -> [Float]
parWorker kData0 points =
  let kData = unClosure kData0
  in map (kData +) points

testFunction = do
  ns <- allNodes
  let arg = toClosure (0.0 :: Float)
  let args = [0]
  pushMapNF ns $(mkClosure [| parWorker arg |]) [args, args, args]

main = do
  register declareStatic
  MPI.defaultWithMPI $ do
    runParIO_ defaultRTSConf $ do
      xs <- testFunction
      io $ print xs
      return ()

[PatrickMaier]

Sorry for the late reply; this email's escaped my attention for months.

The problem is the lack of static support in GHC and the way HdpH emulates
this. This means that every static function must be explicitly declared,
including the indexed static functions of classes ToClosure and ForceCC.

The problem in your code is that you actually use two different instances
of ToClosure, one for [Float] and one for [(Float,Float)] but you derive
them using a single polymorphic instance (which means a single index for
both).

HdpH needs a one-to-one correspondence between the "instance ToClosure
where ..." declaration and the corresponding line "declare
(staticToClosure :: StaticToClosure )". I'd be happy to hear
suggestions how this can be enforced by the type system or further TH
trickery.

On Thu, Oct 24, 2013 at 9:00 PM, cirodrig notifications@github.com wrote:

Parallel HdpH code fails when using polymorphic list instances of
ToClosure and ForceCC. It fails when a closure environment is deserialized.
It appears to fail because the wrong static index of a polymorphic function
is used.

The following code uses a trivial parallel loop with parMapNF to
demonstrate the problem. Switching from polymorphic to monomorphic
instances (by changing #if 1 to #if 0) makes the problem go away.

{-# LANGUAGE CPP, TemplateHaskell, FlexibleInstances,
FlexibleContexts, ScopedTypeVariables #-}

import Control.DeepSeq
import Control.Monad
import Control.Parallel.HdpH hiding(declareStatic)
import Control.Parallel.HdpH.Strategies hiding(declareStatic)
import qualified Control.Parallel.HdpH as HdpH (declareStatic)
import qualified Control.Parallel.HdpH.Strategies as HdpH.Strategies (declareSt
atic)
import qualified Control.Parallel.MP.MPI_ByteString as MPI
import Data.Bits
import Data.List
import Data.Monoid
import Data.Serialize

declareStatic =
mconcat
[ declare $(static 'parWorker)
, HdpH.declareStatic
, HdpH.Strategies.declareStatic
, declare (staticToClosure :: StaticToClosure Float)
, declare (staticForceCC :: StaticForceCC Float)

-- For a crash, these declarations must come first
, declare (staticToClosure :: StaticToClosure [(Float, Float)])
, declare (staticForceCC :: StaticForceCC [(Float, Float)])

-- and these must come afterward
, declare (staticToClosure :: StaticToClosure [Float])
, declare (staticForceCC :: StaticForceCC [Float])
]

instance ToClosure Float where locToClosure = $(here)
instance ForceCC Float where locForceCC = $(here)

instance ToClosure (Float, Float) where locToClosure = $(here)
instance ForceCC (Float, Float) where locForceCC = $(here)

#if 1
-- Causes a crash
instance (ToClosure a, NFData a) => ToClosure [a] where locToClosure = $(here)
instance (ToClosure a, NFData a) => ForceCC [a] where locForceCC = $(here)

#else
-- Doesn't crash
instance ToClosure [(Float, Float)] where locToClosure = $(here)
instance ForceCC [(Float, Float)] where locForceCC = $(here)

instance ToClosure [Float] where locToClosure = $(here)
instance ForceCC [Float] where locForceCC = $(here)
#endif

parWorker :: Closure Float -> [Float] -> [Float]
parWorker kData0 points =
let kData = unClosure kData0
in map (kData +) points

testFunction = do
ns <- allNodes
let arg = toClosure (0.0 :: Float)
let args = [0]
pushMapNF ns $(mkClosure [| parWorker arg |]) [args, args, args]

main = do
register declareStatic
MPI.defaultWithMPI $ do
runParIO_ defaultRTSConf $ do
xs <- testFunction
io $ print xs
return ()

Reply to this email directly or view it on GitHubhttps://github.com//issues/1
.