It is well acknowledged in the AI community that finding a good reward function for reinforcement learning is extremely challenging. Apprenticeship learning (AL) is a class of “learning from demonstration” techniques where the reward function of a Markov Decision Process (MDP) is unknown to the learning agent and the agent uses inverse reinforcement learning (IRL) methods to recover expert policy from a set of expert demonstrations. However, as the agent learns exclusively from observations, given a constraint on the probability of the agent running into unwanted situations, there is no verification, nor guarantee, for the learnt policy on the satisfaction of the restriction. In this dissertation, we study the problem of how to guide AL to learn a policy that is inherently safe while still meeting its learning objective. By combining formal methods with imitation learning, a Counterexample-Guided Apprenticeship Learning algorithm is proposed. We consider a setting where the unknown reward function is assumed to be a linear combination of a set of state features, and the safety property is specified in Probabilistic Computation Tree Logic (PCTL). By embedding probabilistic model checking inside AL, we propose a novel counterexample-guided approach that can ensure both safety and performance of the learnt policy. This algorithm guarantees that given some formal safety specification defined by probabilistic temporal logic, the learnt policy shall satisfy this specification. We demonstrate the effectiveness of our approach on several challenging AL scenarios where safety is essential.

Usability aspects of programming languages are often overlooked, yet have a substantial effect on programmer productivity. These issues are even more acute in the field of Inductive Synthesis, where programs are automatically generated from sample expected input and output data, and the programmer needs to be able to comprehend, and confirm or reject the suggested programs. A promising method of Inductive Synthesis, CombInduce, which is particularly suitable for synthesizing recursive programs, is a candidate for improvements in usability as the target language Combilog is not user-friendly. The method requires the target language to be strictly compositional, hence devoid of variables, yet have the expressiveness of definite clause programs. This sets up a challenging problem for establishing a user-friendly but equally expressive target language. Alternatives to Combilog, such as Quine's Predicate-functor Logic and Schönfinkel and Curry's Combinatory Logic also do not offer a practical notation: finding a more usable representation is imperative. This thesis presents two distinct approaches towards more convenient representations which still maintain compositionality. The first is Visual Combilog (VC), a system for visualizing Combilog programs. In this approach Combilog remains as the target language for synthesis, but programs can be read and modified by interacting with the equivalent diagrams instead. VC is implemented as a split-view editor that maintains the equivalent Combilog and VC representations on-the-fly, automatically transforming them as necessary. The second approach is Combilog with Name Projection (CNP), a textual iteration of Combilog that replaces numeric argument positions with argument names. The result is a language where argument names make the notation more readable, yet compositionality is preserved by avoiding variables. Compositionality is demonstrated by implementing CombInduce with CNP as the target language, revealing that programs with the same level of recursive complexity can be synthesized in CNP equally well, and establishing the underlying method of synthesis can also work with CNP. Our evaluations of the user-friendliness of both representations are supported by a range of methods from Information Visualization, Cognitive Modelling, and Human-Computer Interaction. The increased usability of both representations are confirmed by empirical user studies: an often neglected aspect of language design.