18.5.3 Advanced Hyperparameter Customization

You can build an Unsupervised EdgeWise model using rich hyperparameter customization. Internally for each node, GraphWise applies an aggregation of the representation of neighbors. You can configure this operation through one of the following sub-config classes:

Also, you can enable or disable a graphics processing unit (GPU) by using the enable_ml_accelerators graph server (PGX) configuration parameter (see Configuration Parameters for the Graph Server (PGX) Engine for more information). In addition, ensure that your system meets the following prerequisites to use the GPU support:

  • You must have a GPU device with the CUDA (Compute Unified Device Architecture) toolkit installed.
  • The following list of CUDA libraries are expected:
    • libcuda.so.1
    • libnvrtc.so.12
    • libnvToolsExt.so.1
    • libcudart.so.12

Note that the enable_ml_accelerators option is enabled by default. But if a GPU device is not detected and the CUDA toolkit is not installed, then this feature gets disabled and the CPU will be used for all the PgxML library operations.

The following code examples uses the GraphWiseConvLayerConfig class for the convolutional layer configuration. The examples also specifies a weight decay parameter of 0.001 and dropout with dropping probability 0.5 for the GraphWise model to counteract overfitting. Also, it is recommended to disable normalization of embeddings when you intend to use them in downstream classfication tasks. Note that the setEnableAccelerator method is enabled to use any available GPU. 

opg4j> var weightProperty = analyst.pagerank(trainGraph).getName()
opg4j> var convLayerConfig = analyst.graphWiseConvLayerConfigBuilder().
         setNumSampledNeighbors(25).
         setActivationFunction(ActivationFunction.TANH).
         setWeightInitScheme(WeightInitScheme.XAVIER).
         setWeightedAggregationProperty(weightProperty).
         setDropoutRate(0.5).
         build()
opg4j> var dgiLayerConfig = analyst.graphWiseDgiLayerConfigBuilder().
         setCorruptionFunction(new PermutationCorruption()).
         setDiscriminator(GraphWiseDgiLayerConfig.Discriminator.BILINEAR).
         setReadoutFunction(GraphWiseDgiLayerConfig.ReadoutFunction.MEAN).
         build()
opg4j> var model = analyst.unsupervisedEdgeWiseModelBuilder().
         setVertexInputPropertyNames("vertex_features").
         setEdgeInputPropertyNames("edge_features").
         setConvLayerConfigs(convLayerConfig).
         setDgiLayerConfig(dgiLayerConfig).
         setWeightDecay(0.001).
         setNormalize(false).  // recommended
         setEnableAccelerator(true). // Enable or disable GPU
         build()
String weightProperty = analyst.pagerank(trainGraph).getName();
GraphWiseConvLayerConfig convLayerConfig = analyst.graphWiseConvLayerConfigBuilder()
    .setNumSampledNeighbors(25)
    .setActivationFunction(ActivationFunction.TANH)
    .setWeightInitScheme(WeightInitScheme.XAVIER)
    .setWeightedAggregationProperty(weightProperty)
    .setDropoutRate(0.5)
    .build();

GraphWiseDgiLayerConfig dgiLayerConfig = analyst.graphWiseDgiLayerConfigBuilder()
    .setCorruptionFunction(new PermutationCorruption())
    .setDiscriminator(GraphWiseDgiLayerConfig.Discriminator.BILINEAR)
    .setReadoutFunction(GraphWiseDgiLayerConfig.ReadoutFunction.MEAN)
    .build();

UnsupervisedEdgeWiseModel model = analyst.unsupervisedEdgeWiseModelBuilder()
    .setVertexInputPropertyNames("vertex_features")
    .setEdgeInputPropertyNames("edge_features")
    .setConvLayerConfigs(convLayerConfig)
    .setDgiLayerConfigs(dgiLayerConfig)
    .setWeightDecay(0.001)
    .setNormalize(false)  // recommended
    .setEnableAccelerator(true) // Enable or disable GPU
    .build();
weightProperty = analyst.pagerank(train_graph).name

conv_layer_config = dict(num_sampled_neighbors=25,
                         activation_fn='tanh',
                         weight_init_scheme='xavier',
                         neighbor_weight_property_name=weightProperty,
                         dropout_rate=0.5)

conv_layer = analyst.graphwise_conv_layer_config(**conv_layer_config)

dgi_layer_config = dict(corruption_function=None,
                        readout_function="mean",
                        discriminator="bilinear")

dgi_layer = analyst.graphwise_dgi_layer_config(**dgi_layer_config)

params = dict(conv_layer_config=[conv_layer],
              dgi_layer_config=dgi_layer,
              loss_fn="sigmoid_cross_entropy",
              vertex_input_property_names=["vertex_features"],
              edge_input_property_names=["edge_features"],
              seed=17,
              normalize=False,  # recommended
              weight_decay=0.001,
              enable_accelerator=True) # Enable or disable GPU
)

model = analyst.unsupervised_edgewise_builder(**params)

In the preceding example, you can replace GraphWiseConvLayerConfig with the GraphWiseAttentionLayerConfig class to build a graph attention network model. Also, note that if the number of sampled neighbors is set to -1 using setNumSampledNeighbors, then all the neighboring nodes will be sampled. 

opg4j> var convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder().
         setNumSampledNeighbors(25).
         setActivationFunction(ActivationFunction.LEAKY_RELU).
         setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM).
         setHeadAggregation(AggregationOperation.MEAN).
         setNumHeads(4).
         setDropoutRate(0.5).
         build()
GraphWiseAttentionLayerConfig convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder()
    .setNumSampledNeighbors(25)
    .setActivationFunction(ActivationFunction.LEAKY_RELU)
    .setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM)
    .setHeadAggregation(AggregationOperation.MEAN)
    .setNumHeads(4)
    .setDropoutRate(0.5)
    .build();
conv_layer_config = dict(num_sampled_neighbors=25,
                         activation_fn='leaky_relu',
                         weight_init_scheme='xavier_uniform',
                         aggregation_operation='mean',
                         num_heads=4,
                         dropout_rate=0.5)
See the Javadoc for more information.

Parent topic: Using the Unsupervised EdgeWise Algorithm