The availability of advanced technical systems such as wind turbines, lithography systems, and X-ray systems, is crucial for the smooth operation of public services as well as for the primary processes of companies. Unavailability – especially when unplanned – and failures of these systems have severe consequences, both from a societal and financial point of view. Maintenance optimization deals with minimizing the risk of such failures, and otherwise mitigate their consequences, by performing preventive replacements on critical components when needed. However, early preventive replacements lead to high capital expenditures as the useful lifetimes of the systems is cut short. The most important challenge is therefore to trade-off two conflicting objectives: (i) minimize the risk of failure and unplanned downtime with all its adverse consequences and (ii) maximize the utilization of the useful lifetime of a component. This challenge is particularly difficult when components’ deterioration processes or lifetime distributions are characterized by parameters that are a-priori unknown and need to be learned. Fortunately, there is an increasing and improved availability of real-time data stemming from ubiquitous sensors installed in modern systems, which provides enormous opportunities when leveraged in maintenance optimization characterized by such parameter uncertainty. In this thesis, the overarching topic is the development of a general theory for a class of maintenance problems characterized by parameter uncertainty. We propose a novel, unified framework in the form of a structured road map to develop mathematical models that integrate learning from real-time data with decision making, and demonstrate its wide applicability by adopting it for the analysis of various scenarios.