18.2.3 Advanced Hyperparameter Customization
You can build a GraphWise model using rich hyperparameter customization. Internally for each node, GraphWise applies an aggregation of the representation of neighbors. You can configure this operation through one of the following sub-config classes:
GraphWiseConvLayerConfig: GraphWiseConvLayer is based on Inductive Representation Learning on Large Graphs (GraphSage) by Hamilton et al.GraphWiseAttentionLayerConfig: GraphWiseAttentionLayer is based on Graph Attention Neworks (GAT) by Velickovic et al. which makes the aggregation smarter but comes with larger computation cost.
The GraphWisePredictionLayerConfig class implements the
prediction layer config.
Also, you can enable or disable a graphics processing unit (GPU) by
using the enable_ml_accelerators graph server (PGX)
configuration parameter (see Configuration Parameters for the Graph Server (PGX) Engine for more information). In addition, ensure that your system meets the
following prerequisites to use the GPU support:
- You must have a GPU device with the CUDA (Compute Unified Device Architecture) toolkit installed.
- The following list of CUDA libraries are expected:
libcuda.so.1libnvrtc.so.12libnvToolsExt.so.1libcudart.so.12
Note that the enable_ml_accelerators option is
enabled by default. But if a GPU device is not detected and the CUDA toolkit is
not installed, then this feature gets disabled and the CPU will be used for all
the PgxML library operations.
The following code examples uses the
GraphWiseConvLayerConfig class for the convolutional layer
configuration. The examples specifies a weight decay parameter of
0.001 and dropout with dropping probability
0.5 for the GraphWise model to counteract overfitting. Also,
note that the setEnableAccelerator method is enabled to use any
available GPU.
opg4j> var weightProperty = analyst.pagerank(trainGraph).getName();
opg4j> var convLayerConfig = analyst.graphWiseConvLayerConfigBuilder().
setNumSampledNeighbors(25).
setActivationFunction(ActivationFunction.TANH).
setWeightInitScheme(WeightInitScheme.XAVIER).
setWeightedAggregationProperty(weightProperty).
setDropoutRate(0.5).
build()
opg4j> var predictionLayerConfig = analyst.graphWisePredictionLayerConfigBuilder().
setHiddenDimension(32).
setActivationFunction(ActivationFunction.RELU).
setWeightInitScheme(WeightInitScheme.HE).
setDropoutRate(0.5).
build()
opg4j> var model = analyst.supervisedGraphWiseModelBuilder().
setVertexInputPropertyNames("vertex_features").
setEdgeInputPropertyNames("edge_features").
setVertexTargetPropertyName("labels").
setConvLayerConfigs(convLayerConfig).
setPredictionLayerConfigs(predictionLayerConfig).
setWeightDecay(0.001).
setNormalize(false).
setEmbeddingDim(256).
setLearningRate(0.05).
setNumEpochs(30).
setSeed(42).
setShuffle(false).
setStandardize(true).
setBatchSize(64).
setEnableAccelerator(true). // Enable or disable GPU
build()String weightProperty = analyst.pagerank(trainGraph).getName();
GraphWiseConvLayerConfig convLayerConfig = analyst.graphWiseConvLayerConfigBuilder()
.setNumSampledNeighbors(25)
.setActivationFunction(ActivationFunction.TANH)
.setWeightInitScheme(WeightInitScheme.XAVIER)
.setWeightedAggregationProperty(weightProperty)
.setDropoutRate(0.5)
.build();
GraphWisePredictionLayerConfig predictionLayerConfig = analyst.graphWisePredictionLayerConfigBuilder()
.setHiddenDimension(32)
.setActivationFunction(ActivationFunction.RELU)
.setWeightInitScheme(WeightInitScheme.HE)
.setDropoutRate(0.5)
.build();
SupervisedGraphWiseModel model = analyst.supervisedGraphWiseModelBuilder()
.setVertexInputPropertyNames("vertex_features")
.setEdgeInputPropertyNames("edge_features")
.setVertexTargetPropertyName("labels")
.setConvLayerConfigs(convLayerConfig)
.setPredictionLayerConfigs(predictionLayerConfig)
.setWeightDecay(0.001)
.setNormalize(false)
.setEmbeddingDim(256)
.setLearningRate(0.05)
.setNumEpochs(30)
.setSeed(42)
.setShuffle(false)
.setStandardize(true)
.setBatchSize(64)
.setEnableAccelerator(true) // Enable or disable GPU
.build();weightProperty = analyst.pagerank(train_graph).name
conv_layer_config = dict(num_sampled_neighbors=25,
activation_fn='tanh',
weight_init_scheme='xavier',
neighbor_weight_property_name=weightProperty,
dropout_rate=0.5)
conv_layer = analyst.graphwise_conv_layer_config(**conv_layer_config)
pred_layer_config = dict(hidden_dim=32,
activation_fn='relu',
weight_init_scheme='he',
dropout_rate=0.5)
pred_layer = analyst.graphwise_pred_layer_config(**pred_layer_config)
params = dict(vertex_target_property_name="labels",
conv_layer_config=[conv_layer],
pred_layer_config=[pred_layer],
vertex_input_property_names=["vertex_features"],
edge_input_property_names=["edge_features"],
seed=17,
weight_decay=0.001,
normalize=false,
layer_size=256,
learning_rate=0.05,
num_epochs=30,
seed=42,
standardize=true,
batch_size=64,
enable_accelerator=True # Enable or disable GPU
)
model = analyst.supervised_graphwise_builder(**params)In the preceding example, you can replace
GraphWiseConvLayerConfig with the
GraphWiseAttentionLayerConfig class to build a graph attention
network model. Also, note that if the number of sampled neighbors is set to
-1 using setNumSampledNeighbors, then all the
neighboring nodes will be sampled.
opg4j> var convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder().
setNumSampledNeighbors(25).
setActivationFunction(ActivationFunction.LEAKY_RELU).
setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM).
setHeadAggregation(AggregationOperation.MEAN).
setNumHeads(4).
setDropoutRate(0.5).
build()GraphWiseAttentionLayerConfig convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder()
.setNumSampledNeighbors(25)
.setActivationFunction(ActivationFunction.LEAKY_RELU)
.setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM)
.setHeadAggregation(AggregationOperation.MEAN)
.setNumHeads(4)
.setDropoutRate(0.5)
.build();conv_layer_config = dict(num_sampled_neighbors=25,
activation_fn='leaky_relu',
weight_init_scheme='xavier_uniform',
aggregation_operation='mean',
num_heads=4,
dropout_rate=0.5)