17.4.3 Advanced Hyperparameter Customization
GraphWiseConvLayerConfig: GraphWiseConvLayer is based on Inductive Representation Learning on Large Graphs (GraphSage) by Hamilton et al.GraphWiseAttentionLayerConfig: GraphWiseAttentionLayer is based on Graph Attention Networks (GAT) by Velickovic et al. which makes the aggregation smarter but comes with larger computation cost.
Also, you can enable or disable a graphics processing unit (GPU) by
using the enable_ml_accelerators graph server (PGX)
configuration parameter (see Configuration Parameters for the Graph Server (PGX) Engine for more information). In addition, ensure that your system meets the
following prerequisites to use the GPU support:
- You must have a GPU device with the CUDA (Compute Unified Device Architecture) toolkit installed.
- The following list of CUDA libraries are expected:
libcuda.so.1libnvrtc.so.12libnvToolsExt.so.1libcudart.so.12
Note that the enable_ml_accelerators option is
enabled by default. But if a GPU device is not detected and the CUDA toolkit is
not installed, then this feature gets disabled and the CPU will be used for all
the PgxML library operations.
The following code examples uses the
GraphWiseConvLayerConfig class for the convolutional layer
configuration. The examples specifies a weight decay parameter of
0.001 and dropout with dropping probability
0.5 for the GraphWise model to counteract overfitting. Also, it
is recommended to disable normalization of embeddings when you intend to use them in
downstream classfication tasks. Note that the setEnableAccelerator
method is enabled to use any available GPU.
opg4j> var weightProperty = analyst.pagerank(trainGraph).getName()
opg4j> var convLayerConfig = analyst.graphWiseConvLayerConfigBuilder().
setNumSampledNeighbors(25).
setActivationFunction(ActivationFunction.TANH).
setWeightInitScheme(WeightInitScheme.XAVIER).
setWeightedAggregationProperty(weightProperty).
setDropoutRate(0.5).
build()
opg4j> var dgiLayerConfig = analyst.graphWiseDgiLayerConfigBuilder().
setCorruptionFunction(new PermutationCorruption()).
setDiscriminator(GraphWiseDgiLayerConfig.Discriminator.BILINEAR).
setReadoutFunction(GraphWiseDgiLayerConfig.ReadoutFunction.MEAN).
build()
opg4j> var model = analyst.unsupervisedGraphWiseModelBuilder().
setVertexInputPropertyNames("vertex_features").
setEdgeInputPropertyNames("edge_features").
setConvLayerConfigs(convLayerConfig).
setDgiLayerConfig(dgiLayerConfig).
setLossFunction(UnsupervisedGraphWiseModelConfig.LossFunction.SIGMOID_CROSS_ENTROPY).
setEmbeddingDim(256).
setLearningRate(0.05).
setNumEpochs(30).
setSeed(42).
setShuffle(false).
setStandardize(true).
setNormalize(false). // recommended
setBatchSize(64).
setEnableAccelerator(true). // Enable or disable GPU
build()String weightProperty = analyst.pagerank(trainGraph).getName();
GraphWiseConvLayerConfig convLayerConfig = analyst.graphWiseConvLayerConfigBuilder()
.setNumSampledNeighbors(25)
.setActivationFunction(ActivationFunction.TANH)
.setWeightInitScheme(WeightInitScheme.XAVIER)
.setWeightedAggregationProperty(weightProperty)
.setDropoutRate(0.5)
.build();
GraphWiseDgiLayerConfig dgiLayerConfig = analyst.graphWiseDgiLayerConfigBuilder()
.setCorruptionFunction(new PermutationCorruption())
.setDiscriminator(GraphWiseDgiLayerConfig.Discriminator.BILINEAR)
.setReadoutFunction(GraphWiseDgiLayerConfig.ReadoutFunction.MEAN)
.build();
UnsupervisedGraphWiseModel model = analyst.unsupervisedGraphWiseModelBuilder()
.setVertexInputPropertyNames("vertex_features")
.setEdgeInputPropertyNames("edge_features")
.setDgiLayerConfig(dgiLayerConfig)
.setLossFunction(UnsupervisedGraphWiseModelConfig.LossFunction.SIGMOID_CROSS_ENTROPY)
.setConvLayerConfigs(convLayerConfig)
.setWeightDecay(0.001)
.setEmbeddingDim(256)
.setLearningRate(0.05)
.setNumEpochs(30)
.setSeed(42)
.setShuffle(false)
.setStandardize(true)
.setNormalize(false) // recommended
.setBatchSize(64)
.setEnableAccelerator(true) // Enable or disable GPU
.build();weightProperty = analyst.pagerank(train_graph).name
conv_layer_config = dict(num_sampled_neighbors=25,
activation_fn='tanh',
weight_init_scheme='xavier',
neighbor_weight_property_name=weightProperty,
dropout_rate=0.5)
conv_layer = analyst.graphwise_conv_layer_config(**conv_layer_config)
dgi_layer_config = dict(corruption_function=None,
readout_function="mean",
discriminator="bilinear")
dgi_layer = analyst.graphwise_dgi_layer_config(**dgi_layer_config)
params = dict(conv_layer_config=[conv_layer],
dgi_layer_config=dgi_layer,
loss_fn="sigmoid_cross_entropy",
vertex_input_property_names=["vertex_features"],
edge_input_property_names=["edge_features"],
weight_decay=0.001,
layer_size=256,
learning_rate=0.05,
num_epochs=30,
seed=42,
standardize=True,
normalize=False, # recommended
batch_size=64,
enable_accelerator=True # Enable or disable GPU
)
model = analyst.unsupervised_graphwise_builder(**params)In the preceding example, you can replace
GraphWiseConvLayerConfig with the
GraphWiseAttentionLayerConfig class to build a graph attention
network model. Also, note that if the number of sampled neighbors is set to
-1 using setNumSampledNeighbors, then all the
neighboring nodes will be sampled.
opg4j> var convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder().
setNumSampledNeighbors(25).
setActivationFunction(ActivationFunction.LEAKY_RELU).
setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM).
setHeadAggregation(AggregationOperation.MEAN).
setNumHeads(4).
setDropoutRate(0.5).
build()GraphWiseAttentionLayerConfig convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder()
.setNumSampledNeighbors(25)
.setActivationFunction(ActivationFunction.LEAKY_RELU)
.setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM)
.setHeadAggregation(AggregationOperation.MEAN)
.setNumHeads(4)
.setDropoutRate(0.5)
.build();conv_layer_config = dict(num_sampled_neighbors=25,
activation_fn='leaky_relu',
weight_init_scheme='xavier_uniform',
aggregation_operation='mean',
num_heads=4,
dropout_rate=0.5)