Scala: Tutorial on building a complex hierarchy of traits and classes
===

I have posted several questions on SO recently dealing with Scala [traits][1], [representation types][2], [member types][3], [manifests][4], and [implicit evidence][5]. Behind these questions is my project to build modeling software for biological protein networks. Despite the immensely helpful answers, which have gotten me closer than I ever could get on my own, I have still not arrived at an solution for my project. A couple of answers have suggested that my design is flawed, which is why the solutions to the `Foo`-framed questions don't work in practice. Here I am posting a more complicated (but still greatly simplified) version of my problem. My hope is that the problem and solution will serve a useful tutorial for people trying to build complex hierarchies of traits and classes in Scala.

The highest-level class in my project is the biological reaction rule. A rule describes how one or two reactants are transformed by a reaction. Each reactant is a graph that has nodes called monomers and edges that connect between named sites on the monomers. Each site also has a state that it can be in. A rule might say something like this: there is one reactant made of monomer A bound to monomer B through sites a1 and b1, respectively; the bond is broken by the rule leaving sites a1 and b1 unbound; simultaneously on monomer A, the state of site a1 is changed from U to P. I would write this as:

    A(a1~U-1).B(b1-1) -> A(a1~P) + B(b1)

(Parsing strings like this in Scala was so easy, it made my head spin.) The `-1` indicates that bond #1 is between those sites--the number is just a arbitrary label.

Here is what I have so far along with the reasoning for why I added each component. It compiles, but only with gratuitous use of `asInstanceOf`. I know I can solve any type problem with `asInstanceOf`, but I might as well go back to Matlab if that's what it takes:

I represent rules with a basic class:

    case class Rule(
      reactants: Seq[ReactantGraph], // The starting monomers and edges
      producedMonomers: Seq[ProducedMonomer], // Only new monomers go here
      producedEdges: Seq[ProducedEdge] // All new edges
    ) {
      // Example method that shows different monomers being combined and down-cast
      def combineIntoOneGraph: Graph = {
        val all_monomers = reactants.flatMap(_.monomers) ++ producedMonomers
        val all_edges = reactants.flatMap(_.edges) ++ producedEdges
        GraphClass(all_monomers, all_edges)
      }
    }

The class for graphs `GraphClass` has type parameters because so that I can put constraints on what kinds of monomers and edges are allowed in a particular graph; for example, there cannot be any `ProducedMonomer`s in the `Reactant` of a `Rule`. I would also like to be able to `collect` all the `Monomer`s of a particular type, say `ReactantMonomer`s. I use type aliases to manage the constraints.

    case class GraphClass[
      +MonomerType <: Monomer,
      +EdgeType <: Edge
    ](
      monomers: Seq[MonomerType],
      edges: Seq[EdgeType]
    ) {
      // Methods that demonstrate the need for a manifest on MonomerClass
      def justTheProductMonomers: Seq[ProductMonomer] = {
        monomers.collect{
          case x if isProductMonomer(x) => x.asInstanceOf[ProductMonomer]
        }
      }
      def isProductMonomer(monomer: Monomer): Boolean = (
        monomer.manifest._1 <:< manifest[ProductStateSite] &&
        monomer.manifest._2 <:< manifest[ProductEdgeSite]
      )
    }

    // The most generic Graph
    type Graph = GraphClass[Monomer, Edge]
    // Anything allowed in a reactant
    type ReactantGraph = GraphClass[ReactantMonomer, ReactantEdge]
    // Anything allowed in a product, which I sometimes extract from a Rule
    type ProductGraph = GraphClass[ProductMonomer, ProductEdge]

The class for monomers `MonomerClass` has type parameters, as well, so that I can put constraints on the sites; for example, a `ConsumedMonomer` cannot have a `StaticSite`. Furthermore, I need to `collect` all the monomers of a particular type to, say, collect all the monomers in a rule that are in the product, so I add a `Manifest` to each type parameter.

    case class MonomerClass[
      +StateSiteType <: StateSite : Manifest,
      +EdgeSiteType <: EdgeSite : Manifest
    ](
      stateSites: Seq[StateSiteType],
      edgeSites: Seq[EdgeSiteType]
    ) {
      type MyType = MonomerClass[StateSiteType, EdgeSiteType]
      def manifest = (
        implicitly[Manifest[_ <: StateSiteType]],
        implicitly[Manifest[_ <: EdgeSiteType]]
      )
      // Method that demonstrates the need for implicit evidence
      // This is where it gets bad
      def replaceSiteWithIntersection[A >: StateSiteType <: ReactantStateSite](
        thisSite: A, // This is a member of this.stateSites
        monomer: ReactantMonomer
      )(
        // Only the sites on ReactantMonomers have the Observed property
        implicit evidence: MyType <:< ReactantMonomer
      ): MyType = {
        val new_this = evidence(this) // implicit evidence usually needs some help
        monomer.stateSites.find(_.name == thisSite.name) match {
          case Some(otherSite) => 
            val newSites = stateSites map {
              case `thisSite` => (
                thisSite.asInstanceOf[StateSiteType with ReactantStateSite]
                .createIntersection(otherSite).asInstanceOf[StateSiteType]
              )
              case other => other
            }
            copy(stateSites = newSites)
          case None => this
        }
      }
    }

    type Monomer = MonomerClass[StateSite, EdgeSite]
    type ReactantMonomer = MonomerClass[ReactantStateSite, ReactantEdgeSite]
    type ProductMonomer = MonomerClass[ProductStateSite, ProductEdgeSite]
    type ConsumedMonomer = MonomerClass[ConsumedStateSite, ConsumedEdgeSite]
    type ProducedMonomer = MonomerClass[ProducedStateSite, ProducedEdgeSite]
    type StaticMonomer = MonomerClass[StaticStateSite, StaticEdgeSite]

My current implementations for the `StateSite` and `EdgeSite` do not have type parameters; they are standard hierarchies of traits, terminating in classes that have a name and some `String`s that represent the appropriate state. (Be nice about using strings to hold object states; they are actually name classes in my real code.) One important purpose of these traits is provide functionality that all the subclasses need, Well, isn't that the purpose of all traits. My traits are special in that many of the methods make small changes to a property of the object that is common to all subclasses of the trait and then return a copy. It would be preferable if the return type matched the underlying type of the object. The lame way to do this is to make all the trait methods abstract, and copy the desired methods into all the subclasses. I am unsure of the Scala way to do this. Some sources suggest a member type `MyType` that stores the underlying type (shown here). Other sources suggest a representation type parameter.

    trait StateSite {
      type MyType <: StateSite 
      def name: String
    }
    trait ReactantStateSite extends StateSite {
      type MyType <: ReactantStateSite
      def observed: Seq[String]
      def stateCopy(observed: Seq[String]): MyType
      def createIntersection(otherSite: ReactantStateSite): MyType = {
        val newStates = observed.intersect(otherSite.observed)
        stateCopy(newStates)
      }
    }
    trait ProductStateSite extends StateSite
    trait ConservedStateSite extends ReactantStateSite with ProductStateSite 
    case class ConsumedStateSite(name: String, consumed: Seq[String]) 
      extends ReactantStateSite {
      type MyType = ConsumedStateSite
      def observed = consumed
      def stateCopy(observed: Seq[String]) = copy(consumed = observed)
    }
    case class ProducedStateSite(name: String, Produced: String)
      extends ProductStateSite 
    case class ChangedStateSite(
      name: String, 
      consumed: Seq[String], 
      Produced: String
    )
      extends ConservedStateSite {
      type MyType = ChangedStateSite
      def observed = consumed
      def stateCopy(observed: Seq[String]) = copy(consumed = observed)
    }
    case class StaticStateSite(name: String, static: Seq[String])
      extends ConservedStateSite {
      type MyType = StaticStateSite
      def observed = static
      def stateCopy(observed: Seq[String]) = copy(static = observed)
    }

The `EdgeSite` class introduces massive complexity into my code that I don't think it is particularly important to understanding Scala. Here is a simplified version that can just be bound or unbound. In reality, `EdgeSite`s also needs to be able to specify something like "this site could be bound to this site or that site, both of which may or may not even be mentioned in this graph"

    trait EdgeSite {def name: String}
    trait ReactantEdgeSite extends EdgeSite
    trait ProductEdgeSite extends EdgeSite
    trait ConservedEdgeSite extends ReactantEdgeSite with ProductEdgeSite

    trait ConsumedEdgeSite extends ReactantEdgeSite
    trait ProducedEdgeSite extends ProductEdgeSite
    trait StaticEdgeSite extends ConservedEdgeSite 

    trait ReactantBoundEdgeSite extends ReactantEdgeSite
    trait ProductBoundEdgeSite extends ProductEdgeSite

    case class ConsumedUnboundEdgeSite(name: String) 
      extends ConsumedEdgeSite 
    case class ConsumedBoundEdgeSite(name: String) 
      extends ConsumedEdgeSite with ReactantBoundEdgeSite

    case class ProducedUnboundEdgeSite(name: String) 
      extends ProducedEdgeSite 
    case class ProducedBoundEdgeSite(name: String) 
      extends ProducedEdgeSite with ProductBoundEdgeSite

    case class ChangedUnboundEdgeSiteToBound(name: String) 
      extends ConservedEdgeSite with ProductBoundEdgeSite
    case class ChangedBoundEdgeSiteToUnbound(name: String) 
      extends ConservedEdgeSite with ReactantBoundEdgeSite

    case class StaticUnboundEdgeSite(name: String) 
      extends StaticEdgeSite 
    case class StaticBoundEdgeSite(name: String) 
      extends StaticEdgeSite with ReactantBoundEdgeSite with ProductBoundEdgeSite

Finally, a simple class for the `BoundEdge`s holds pointers to two bound edge sites. These objects are attached to the `Graph`s to indicate how the `Monomer`s, which are also part of the `Graph`s, are connected.

    trait Edge {def targets: (EdgeSite, EdgeSite)}
    trait ReactantEdge {def targets: (ReactantBoundEdgeSite, ReactantBoundEdgeSite)}
    trait ProductEdge {def targets: (ProductBoundEdgeSite, ProductBoundEdgeSite)}

    case class ConsumedEdge(targets: (ReactantBoundEdgeSite, ReactantBoundEdgeSite))
      extends ReactantEdge 
    case class ProducedEdge(targets: (ProductBoundEdgeSite, ProductBoundEdgeSite))
      extends ProductEdge 
    case class StaticEdge(targets: (StaticBoundEdgeSite, StaticBoundEdgeSite))
      extends ReactantEdge with ProductEdge

My biggest problems are with methods framed like `MonomerClass.replaceSiteWithIntersection`. A lot of methods do some complicated search for particular members of the class, then pass those members to other functions where complicated changes are made to them and return a copy, which then replaces the original in a copy of the higher-level object. Scala is distinctly unhappy with passing instances of a type or member parameter around because of two main reasons that I can see: (1) the covariant type parameter ends up as input to any method that takes them as input, and (2) it is difficult to convince Scala that a method that returns a copy indeed returns an object with exactly the same type as was put in.

I have undoubtedly left some things that will not be clear to everyone. If there are any details I need to add, or excess details I need to delete, I will try to be quick to clear things up.


  [1]: http://stackoverflow.com/questions/11618915
  [2]: http://stackoverflow.com/questions/11277656
  [3]: http://stackoverflow.com/questions/11236122
  [4]: http://stackoverflow.com/questions/11284538/
  [5]: http://stackoverflow.com/questions/11588014