
Sampleefficient proper PAC learning with approximate differential privacy
In this paper we prove that the sample complexity of properly learning a...
read it

AverageCase Information Complexity of Learning
How many bits of information are revealed by a learning algorithm for a ...
read it

Enumeration in Incremental FPTTime
In this paper, we study the relationship of parametrised enumeration com...
read it

Tight Bounds for Collaborative PAC Learning via Multiplicative Weights
We study the collaborative PAC learning problem recently proposed in Blu...
read it

Learnability Can Be Independent of ZFC Axioms: Explanations and Implications
In BenDavid et al.'s "Learnability Can Be Undecidable," they prove an i...
read it

Where is the Information in a Deep Neural Network?
Whatever information a Deep Neural Network has gleaned from past data is...
read it

PACLearning for Strategic Classification
Machine learning (ML) algorithms may be susceptible to being gamed by in...
read it
A Direct Sum Result for the Information Complexity of Learning
How many bits of information are required to PAC learn a class of hypotheses of VC dimension d? The mathematical setting we follow is that of Bassily et al. (2018), where the value of interest is the mutual information I(S;A(S)) between the input sample S and the hypothesis outputted by the learning algorithm A. We introduce a class of functions of VC dimension d over the domain X with information complexity at least Ω(dX/d) bits for any consistent and proper algorithm (deterministic or random). Bassily et al. proved a similar (but quantitatively weaker) result for the case d=1. The above result is in fact a special case of a more general phenomenon we explore. We define the notion of information complexity of a given class of functions H. Intuitively, it is the minimum amount of information that an algorithm for H must retain about its input to ensure consistency and properness. We prove a direct sum result for information complexity in this context; roughly speaking, the information complexity sums when combining several classes.
READ FULL TEXT
Comments
There are no comments yet.