Mask Access Logs and Activity Report to Comply with Privacy Laws

Use the scripts in this section to automate the process of masking information in the Activity Report or Access Logs to comply with privacy laws and to, optionally, email the report to a recipient.

Because of the stringent privacy laws of some countries, the information available in the Activity Reports and Access Logs may have to be hidden from Service Administrators to protect privacy of users.

You use anonymizeData.bat to mask information in the Activity Report or Access Logs to comply with privacy laws and to, optionally, email it. To mask information, schedule this script or a variation there of using Windows scheduler so that it runs everyday soon after the daily maintenance process for each environment is complete.

Use these information sources:

You manually create anonymizeData.bat by copying the Windows script provided in the following procedure and schedule it using Windows scheduler. You may create and run similar platform-appropriate scripts if you are not using Windows for scheduling.

anonymizeData.bat is a wrapper script, which executes the anonymizeData.ps1 script, which you create and update as explained in the following procedure.

Note:

If you are using the PDF version of this document: To avoid line breaks and footer information that will render these scripts unusable, copy them from the HTML version of this topic.

If the password you use contains special characters, see Handling Special Characters

  1. Create a batch (BAT) file named anonymizeData.bat containing the following script and save it in a convenient location, for example, C:\automate_scripts.
    @echo off
    set paramRequiredMessage=Syntax: anonymizeData.bat USERNAME PASSWORD/PASSWORD_FILE URL [EMAIL_TO_ADDRESS]
    
    if "%~1" == "" (
      echo User Name is missing.
      echo %paramRequiredMessage%
      exit /b 1
      )
    if "%~2" == "" (
      echo Password or Password_File is missing.
      echo %paramRequiredMessage%
      exit /b 1
      )
    if "%~3" == "" (
      echo URL is missing.
      echo %paramRequiredMessage%
      exit /b 1
      )
    
    PowerShell.exe -File anonymizeData.ps1 %*
  2. Create a PowerShell script (PS1) file named anonymizeData.ps1 containing the following script and save it in a convenient location, for example, C:\automate_scripts.
    # Anonymize data script
    
    $username=$args[0]
    $password=$args[1]
    $url=$args[2]
    $emailtoaddress=$args[3]
    
    # Generic variables
    $date=$(get-date -f dd_MM_yy_HH_mm_ss)
    $datedefaultformat=$(get-date)
    $logdir="./logs/"
    $logfile="$logdir/anonymize-data-" + $date + ".log"
    $filelist="filelist.txt"
    
    function LogMessage
    {
      $message=$args[0]
    
      echo "$message" >> $logfile
    }
    
    function EchoAndLogMessage
    {
      $message=$args[0]
    
      echo "$message"
      echo "$message" >> $logfile
    }
    function Init
    {
       $logdirexists=Test-Path $logdir
       if (!($logdirexists)) {
           mkdir $logdir 2>&1 | out-null
        }
    
       $logfileexists=Test-Path $logfile
       if ($logfileexists) {
           rm $logfile 2>&1 | out-null
        }
    
       $filelistexists=Test-Path $filelist
       if ($filelistexists) {
           rm $filelist 2>&1 | out-null
        }
    }
    
    function ProcessCommand
    {
       $op=$args
       echo "EPM Automate operation: epmautomate.bat $op" >> $logfile
       if ($op -eq 'listfiles') {
          epmautomate.bat $op | where {$_ -like ' apr/*/access_log.zip'} | Tee-Object -FilePath $filelist | Out-File $logfile -Append 2>&1
        } else {
          epmautomate.bat $op >> $logfile 2>&1
            if ($LASTEXITCODE -ne 0) {
               echo "EPM Automate operation failed: epmautomate.bat $op. See $logfile for details."
               #exit
               }
        }
    }
    
    function RunEpmAutomateCommands
    {
        EchoAndLogMessage "Running EPM Automate commands to anonymize data in the access logs and activity reports."
        ProcessCommand login $username $password $url
        ProcessCommand listfiles
        ProcessFiles
        ProcessCommand logout
    }
    
    function ProcessActivityReport
    {
        $activityreport=$args[0]
        $user=$args[1]
    
        $activityreportexists=Test-Path "$activityreport"
        if ($activityreportexists) {
            LogMessage "Removing User ID: $user from activity report $activityreport"
            (Get-Content "$activityreport").replace("$user", 'XXXXX') | Set-Content "$activityreport"
            $txt = [io.file]::ReadAllText("$activityreport") -replace "`r`n","`n"
            [io.file]::WriteAllText("$activityreport", $txt)
            #Get-ChildItem -File -Recurse | % { $x = get-content -raw -path $activityreport; $x -replace "`r`n","`n" | set-content -path $activityreport }
        }
    }
    
    function AnonymizeData
    {
        $aprdir=$args[0]
        $datestampdir=$args[1]
        $path="$aprdir/$datestampdir"
        $accesslogzipped="access_log.zip"
        $accesslog="access_log.csv"
        $accesslogupdated=$accesslog + ".tmp"
        $activityreportfile="$datestampdir" + ".html"
        $userArray = @()
    
        expand-Archive -Path "$path/$accesslogzipped" -DestinationPath $path
        rm $path/$accesslogzipped 2>&1 | out-null
        $accesslogexists=Test-Path "$path/$accesslog"
        if ($accesslogexists) {
            EchoAndLogMessage "Processing access log: $path/$accesslog"
            Get-Content $path/$accesslog | ForEach-Object {
                $elements=[regex]::Split( $_ , ',(?=(?:[^"]|"[^"]*")*$)' )
                $date=$elements[0]
                $time=$elements[1]
                $uri=$elements[2]
                $duration=$elements[3]
                $bytes=$elements[4]
                $ip=$elements[5]
                $user=$elements[6]
                $screen=$elements[7]
                $action=$elements[8]
                $object=$elements[9]
                if ($date -like 'Date') {
                    echo "$_" >> $path/$accesslogupdated
                } else {
                    if ($user -notlike '-') {
                        LogMessage "Removing instance of User ID: $user from $path/$accesslog."
                        echo "$date,$time,$uri,$duration,$bytes,$ip,XXXXX,$screen,$action,$object" >> $path/$accesslogupdated
                        $userArray += $user
                    } else {
                        echo "$date,$time,$uri,$duration,$bytes,$ip,$user,$screen,$action,$object" >> $path/$accesslogupdated
                    }
                }
            }
            #Get-ChildItem -File -Recurse | % { $x = get-content -raw -path $path/$accesslogupdated; $x -replace "`r`n","`n" | set-content -path $path/$accesslogupdated }
            $txt = [io.file]::ReadAllText("$path/$accesslogupdated") -replace "`r`n","`n"
            [io.file]::WriteAllText("$path/$accesslogupdated", $txt)
            mv -Force $path/$accesslogupdated $path/$accesslog
            Compress-Archive -Path $path/$accesslog $path/$accesslogzipped
            rm $path/$accesslog 2>&1 | out-null
        }
    
        EchoAndLogMessage "Processing activity report: $path/$activityreportfile"
        $userArray = $userArray | Select-Object -Unique
        foreach ($element in $userArray) {
            ProcessActivityReport "$path/$activityreportfile" "$element"            
        }
    }
    
    function ProcessFiles
    {
        # Loop through iteration csv file and parse
        Get-Content $filelist | ForEach-Object {
            $fullpath=$_.trim()
            $elements=$fullpath.split('/')
            $aprdir=$elements[0]
            $datestampdir=$elements[1]
            $accesslogfile="access_log.zip"
            $activityreportfile="$datestampdir" + ".html"
            $datestampdirexists=Test-Path "$aprdir/$datestampdir"
            $accesslog="$aprdir/$datestampdir/$accesslogfile"
            $activityreport="$aprdir/$datestampdir/$activityreportfile"
    
            echo "fullpath: $fullpath" >> $logfile
            echo "aprdir: $aprdir, datestampdir: $datestampdir" >> $logfile
            if (!($datestampdirexists)) {
                mkdir "$aprdir/$datestampdir" -ea 0 2>&1 | out-null
                ProcessCommand downloadfile "$accesslog"
                ProcessCommand downloadfile "$activityreport"
                mv "$accesslogfile" "$aprdir/$datestampdir"
                mv "$activityreportfile" "$aprdir/$datestampdir"
                AnonymizeData "$aprdir" "$datestampdir"
                ProcessCommand deletefile "$accesslog"
                ProcessCommand deletefile "$activityreport"
                ProcessCommand uploadfile "$accesslog" "$aprdir/$datestampdir"
                ProcessCommand uploadfile "$activityreport" "$aprdir/$datestampdir"
            } else {
                EchoAndLogMessage "Files in directory $aprdir/$datestampdir were processed earlier. Skipping these files."
            }
        }
    }
    
    function callSendMail
    {
        $elements=$logfile.split('/')
        $logfilename=$elements[3]
    
        if (${emailtoaddress} -match "@") {
            epmautomate.bat login ${username} ${password} ${url}
            epmautomate.bat uploadFile "$logfile"
            epmautomate.bat sendMail $emailtoaddress "Mask Access Logs and Activity Reports results" Body="The results of running the Mask Access Logs and Activity Reports script are attached." Attachments=$logfilename
            epmautomate.bat deleteFile "$logfilename"
            epmautomate.bat logout
        }
    }
    
    Init
    EchoAndLogMessage "Starting the anonymize data script"
    RunEpmAutomateCommands
    EchoAndLogMessage "Anonymize data script completed"
    EchoAndLogMessage "Refer to logfile: $logfile for details."
    callSendMail
    
  3. Using Windows Scheduler, schedule anonymizeData.bat. See Automating Script Execution for detailed steps.
    You need to supply the following parameter values to execute anonymizeData.bat
    • User name of a Service Administrator
    • Password of the Service Administrator or the location where the encrypted password file is available
    • URL of the service environment in which the Access Logs and Activity Reports are to be masked
    • Optional: The email address to which the report is to be sent. The report is emailed only if this value is specified.