17.2.3 Advanced Hyperparameter Customization

You can build a GraphWise model using rich hyperparameter customization. Internally for each node, GraphWise applies an aggregation of the representation of neighbors. You can configure this operation through one of the following sub-config classes:

• GraphWiseConvLayerConfig: GraphWiseConvLayer is based on Inductive Representation Learning on Large Graphs (GraphSage) by Hamilton et al.
• GraphWiseAttentionLayerConfig: GraphWiseAttentionLayer is based on Graph Attention Neworks (GAT) by Velickovic et al. which makes the aggregation smarter but comes with larger computation cost.

The GraphWisePredictionLayerConfig class implements the prediction layer config.

The following code examples uses the GraphWiseConvLayerConfig class for the convolutional layer configuration. The examples also specifies a weight decay parameter of 0.001 and dropout with dropping probability 0.5 for the GraphWise model to counteract overfitting.

• JShell
• Java
• Python

opg4j> var weightProperty = analyst.pagerank(trainGraph).getName();
opg4j> var convLayerConfig = analyst.graphWiseConvLayerConfigBuilder().
         setNumSampledNeighbors(25).
         setActivationFunction(ActivationFunction.TANH).
         setWeightInitScheme(WeightInitScheme.XAVIER).
         setWeightedAggregationProperty(weightProperty).
         setDropoutRate(0.5).
         build()
opg4j> var predictionLayerConfig = analyst.graphWisePredictionLayerConfigBuilder().
         setHiddenDimension(32).
         setActivationFunction(ActivationFunction.RELU).
         setWeightInitScheme(WeightInitScheme.HE).
         setDropoutRate(0.5).
         build()
opg4j> var model = analyst.supervisedGraphWiseModelBuilder().
         setVertexInputPropertyNames("vertex_features").
         setEdgeInputPropertyNames("edge_features").
         setVertexTargetPropertyName("labels").
         setConvLayerConfigs(convLayerConfig).
         setPredictionLayerConfigs(predictionLayerConfig).
         setWeightDecay(0.001).
         setNormalize(false).
         setEmbeddingDim(256).
         setLearningRate(0.05).
         setNumEpochs(30).
         setSeed(42).
         setShuffle(false).
         setStandardize(true).
         setBatchSize(64).
         build()

String weightProperty = analyst.pagerank(trainGraph).getName();
GraphWiseConvLayerConfig convLayerConfig = analyst.graphWiseConvLayerConfigBuilder()
    .setNumSampledNeighbors(25)
    .setActivationFunction(ActivationFunction.TANH)
    .setWeightInitScheme(WeightInitScheme.XAVIER)
    .setWeightedAggregationProperty(weightProperty)
    .setDropoutRate(0.5)
    .build();

GraphWisePredictionLayerConfig predictionLayerConfig = analyst.graphWisePredictionLayerConfigBuilder()
    .setHiddenDimension(32)
    .setActivationFunction(ActivationFunction.RELU)
    .setWeightInitScheme(WeightInitScheme.HE)
    .setDropoutRate(0.5)
    .build();

SupervisedGraphWiseModel model = analyst.supervisedGraphWiseModelBuilder()
    .setVertexInputPropertyNames("vertex_features")
    .setEdgeInputPropertyNames("edge_features")
    .setVertexTargetPropertyName("labels")
    .setConvLayerConfigs(convLayerConfig)
    .setPredictionLayerConfigs(predictionLayerConfig)
    .setWeightDecay(0.001)
    .setNormalize(false)
    .setEmbeddingDim(256)
    .setLearningRate(0.05)
    .setNumEpochs(30)
    .setSeed(42)
    .setShuffle(false)
    .setStandardize(true)
    .setBatchSize(64)
    .build();

weightProperty = analyst.pagerank(train_graph).name

conv_layer_config = dict(num_sampled_neighbors=25,
                         activation_fn='tanh',
                         weight_init_scheme='xavier',
                         neighbor_weight_property_name=weightProperty,
                         dropout_rate=0.5)

conv_layer = analyst.graphwise_conv_layer_config(**conv_layer_config)

pred_layer_config = dict(hidden_dim=32,
                         activation_fn='relu',
                         weight_init_scheme='he',
                         dropout_rate=0.5)

pred_layer = analyst.graphwise_pred_layer_config(**pred_layer_config)

params = dict(vertex_target_property_name="labels",
              conv_layer_config=[conv_layer],
              pred_layer_config=[pred_layer],
              vertex_input_property_names=["vertex_features"],
              edge_input_property_names=["edge_features"],
              seed=17,
              weight_decay=0.001,
              normalize=false,
              layer_size=256,
              learning_rate=0.05,
              num_epochs=30,
              seed=42,
              standardize=true,
              batch_size=64
)

model = analyst.supervised_graphwise_builder(**params)

In the preceding example, you can replace GraphWiseConvLayerConfig with the GraphWiseAttentionLayerConfig class to build a graph attention network model. Also, note that if the number of sampled neighbors is set to -1 using setNumSampledNeighbors, then all the neighboring nodes will be sampled.

• JShell
• Java
• Python

opg4j> var convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder().
         setNumSampledNeighbors(25).
         setActivationFunction(ActivationFunction.LEAKY_RELU).
         setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM).
         setHeadAggregation(AggregationOperation.MEAN).
         setNumHeads(4).
         setDropoutRate(0.5).
         build()

GraphWiseAttentionLayerConfig convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder()
    .setNumSampledNeighbors(25)
    .setActivationFunction(ActivationFunction.LEAKY_RELU)
    .setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM)
    .setHeadAggregation(AggregationOperation.MEAN)
    .setNumHeads(4)
    .setDropoutRate(0.5)
    .build();

conv_layer_config = dict(num_sampled_neighbors=25,
                         activation_fn='leaky_relu',
                         weight_init_scheme='xavier_uniform',
                         aggregation_operation='mean',
                         num_heads=4,
                         dropout_rate=0.5)

See the Javadoc for more information.