17.5.3 Advanced Hyperparameter Customization
GraphWiseConvLayerConfig
: GraphWiseConvLayer is based on Inductive Representation Learning on Large Graphs (GraphSage) by Hamilton et al.GraphWiseAttentionLayerConfig
: GraphWiseAttentionLayer is based on Graph Attention Networks (GAT) by Velickovic et al. which makes the aggregation smarter but comes with larger computation cost.
Also, you can enable or disable a graphics processing unit (GPU) by
using the enable_ml_accelerators
graph server (PGX)
configuration parameter (see Configuration Parameters for the Graph Server (PGX) Engine for more information). In addition, ensure that your system meets the
following prerequisites to use the GPU support:
- You must have a GPU device with the CUDA (Compute Unified Device Architecture) toolkit installed.
- The following list of CUDA libraries are expected:
libcuda.so.1
libnvrtc.so.12
libnvToolsExt.so.1
libcudart.so.12
Note that the enable_ml_accelerators
option is
enabled by default. But if a GPU device is not detected and the CUDA toolkit is
not installed, then this feature gets disabled and the CPU will be used for all
the PgxML library operations.
The following code examples uses the
GraphWiseConvLayerConfig
class for the convolutional layer
configuration. The examples also specifies a weight decay parameter of
0.001
and dropout with dropping probability
0.5
for the GraphWise model to counteract overfitting.
Also, it is recommended to disable normalization of embeddings when you intend
to use them in downstream classfication tasks. Note that the
setEnableAccelerator
method is enabled to use any available
GPU.
opg4j> var weightProperty = analyst.pagerank(trainGraph).getName()
opg4j> var convLayerConfig = analyst.graphWiseConvLayerConfigBuilder().
setNumSampledNeighbors(25).
setActivationFunction(ActivationFunction.TANH).
setWeightInitScheme(WeightInitScheme.XAVIER).
setWeightedAggregationProperty(weightProperty).
setDropoutRate(0.5).
build()
opg4j> var dgiLayerConfig = analyst.graphWiseDgiLayerConfigBuilder().
setCorruptionFunction(new PermutationCorruption()).
setDiscriminator(GraphWiseDgiLayerConfig.Discriminator.BILINEAR).
setReadoutFunction(GraphWiseDgiLayerConfig.ReadoutFunction.MEAN).
build()
opg4j> var model = analyst.unsupervisedEdgeWiseModelBuilder().
setVertexInputPropertyNames("vertex_features").
setEdgeInputPropertyNames("edge_features").
setConvLayerConfigs(convLayerConfig).
setDgiLayerConfig(dgiLayerConfig).
setWeightDecay(0.001).
setNormalize(false). // recommended
setEnableAccelerator(true). // Enable or disable GPU
build()
String weightProperty = analyst.pagerank(trainGraph).getName();
GraphWiseConvLayerConfig convLayerConfig = analyst.graphWiseConvLayerConfigBuilder()
.setNumSampledNeighbors(25)
.setActivationFunction(ActivationFunction.TANH)
.setWeightInitScheme(WeightInitScheme.XAVIER)
.setWeightedAggregationProperty(weightProperty)
.setDropoutRate(0.5)
.build();
GraphWiseDgiLayerConfig dgiLayerConfig = analyst.graphWiseDgiLayerConfigBuilder()
.setCorruptionFunction(new PermutationCorruption())
.setDiscriminator(GraphWiseDgiLayerConfig.Discriminator.BILINEAR)
.setReadoutFunction(GraphWiseDgiLayerConfig.ReadoutFunction.MEAN)
.build();
UnsupervisedEdgeWiseModel model = analyst.unsupervisedEdgeWiseModelBuilder()
.setVertexInputPropertyNames("vertex_features")
.setEdgeInputPropertyNames("edge_features")
.setConvLayerConfigs(convLayerConfig)
.setDgiLayerConfigs(dgiLayerConfig)
.setWeightDecay(0.001)
.setNormalize(false) // recommended
.setEnableAccelerator(true) // Enable or disable GPU
.build();
weightProperty = analyst.pagerank(train_graph).name
conv_layer_config = dict(num_sampled_neighbors=25,
activation_fn='tanh',
weight_init_scheme='xavier',
neighbor_weight_property_name=weightProperty,
dropout_rate=0.5)
conv_layer = analyst.graphwise_conv_layer_config(**conv_layer_config)
dgi_layer_config = dict(corruption_function=None,
readout_function="mean",
discriminator="bilinear")
dgi_layer = analyst.graphwise_dgi_layer_config(**dgi_layer_config)
params = dict(conv_layer_config=[conv_layer],
dgi_layer_config=dgi_layer,
loss_fn="sigmoid_cross_entropy",
vertex_input_property_names=["vertex_features"],
edge_input_property_names=["edge_features"],
seed=17,
normalize=False, # recommended
weight_decay=0.001,
enable_accelerator=True) # Enable or disable GPU
)
model = analyst.unsupervised_edgewise_builder(**params)
In the preceding example, you can replace
GraphWiseConvLayerConfig
with the
GraphWiseAttentionLayerConfig
class to build a graph attention
network model. Also, note that if the number of sampled neighbors is set to
-1
using setNumSampledNeighbors
, then all the
neighboring nodes will be sampled.
opg4j> var convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder().
setNumSampledNeighbors(25).
setActivationFunction(ActivationFunction.LEAKY_RELU).
setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM).
setHeadAggregation(AggregationOperation.MEAN).
setNumHeads(4).
setDropoutRate(0.5).
build()
GraphWiseAttentionLayerConfig convLayerConfig = analyst.graphWiseAttentionLayerConfigBuilder()
.setNumSampledNeighbors(25)
.setActivationFunction(ActivationFunction.LEAKY_RELU)
.setWeightInitScheme(WeightInitScheme.XAVIER_UNIFORM)
.setHeadAggregation(AggregationOperation.MEAN)
.setNumHeads(4)
.setDropoutRate(0.5)
.build();
conv_layer_config = dict(num_sampled_neighbors=25,
activation_fn='leaky_relu',
weight_init_scheme='xavier_uniform',
aggregation_operation='mean',
num_heads=4,
dropout_rate=0.5)
Parent topic: Using the Unsupervised EdgeWise Algorithm