Logicworks

Volume management of Apple devices

MDM

Apple devices management revolves around MDM solutions and Apple Business Manager portal. We mostly deal with VMware Workspace ONE, SimpleMDM and Jamf Pro products, mainly due to their macOS management capabilities. We also helped customers with less progressive solutions such as Microsoft Intune, Mobile iron, Hexnode and of course the rather infamous Profile Manager from Apple.

For one of our clients, I prepared a detailed comparative study, in which I compared VMware Workspace ONE, Jamf Pro and Microsoft Intune.

Although VMware Workspace ONE UEM uses Munki for some of the macOS software installs, it is more practical for some customers to run standalone Muni infrastructure. I created the synchronization tool wso-munki-manifest-sync, which uses the WSO UEM API to find out the bonds between computers, groups and users and synchronizes them into the structure of file manifests in the Munki repository.

Automated Device Enrollment

Automated Device Enrollment (formerly better known as DEP – Device Enrollment Program) is the highlight of recent Apple device deployment. Users unpack a new Mac, iPad or iPhone from the box. It will automatically enroll itself into corporate management solution, which provides everything they need to start working. The combination of ADE and MDM is a straightforward matter. However, not necessarily in the case of macOS.

Because Macs can be managed with tools other than MDM and generally offer greater flexibility in customization, in some cases, additional tools may need to be installed during the enrollment process. This concept is sometimes called custom DEP. It is based on a digitally signed package sent from MDM to the client. This package can include any kind of software for the macOS.

The very popular installapplications tool provides the server side with a clearly defined sequence of steps to be performed on the client. Communication with the user can be solved using windows or notifications set through DEPNotify. The onboarding guide Octory also may be part of the process.

Open source tools

For effective macOS management, MDM capabilities are not always sufficient or may not be the best choice for all tasks needed to be done.

At Logicworks, I run the Munki infrastructure for our customers. Install packages are delivered using a web server from multiple nodes, synchronized with each other by the unison tool. Clients select a node using DNS load balancing. The connection is always secured with TLS 1.2 or 1.3. The Munki repository shared among multiple customers requires the separation of some files at the web server level. This is accomplished by web request authentication. File downloads can be significantly accelerated by using cache web servers on customers’ local networks and HTTP redirection. File downloads can be significantly accelerated by using cache web servers on customers’ local networks and HTTP redirection. Repository modifications take place over the network from virtualized macOS, which accesses data via the encrypted SMB3 protocol (Samba server).

AutoPkg takes care of the automatic download of the latest versions of software installation packages. It runs once a day and reports changes by email to the company’s Slack channel. Some of the recipes we use are available in my autopkg repository MichalMMac-recipes.

We create our own installation packages for macOS mostly with the munki-pkg tool. Our engineers use installation packages to configure Munki in Mac clients which are not enrolled in MDM. In order to get a digitally signed and notarized package, I extended munki-pkg with the ability to upload packages to the Apple notary service. Due to the notarization requirements, Python 3, which is bundled with Munki needed to be re-signed with our developer certificate.

We monitor Macs using the MunkiReport web application. You can find in our products under the name Sensei MacReport. I took over munkireport-parser script created by my colleague and fundamentally redesigned it. Script uses API to get current data from MunkiReport. Detected problems on devices are sent to our technicians via email.

If you need to use macOS with the Active Directory, we recommend loose integration. We achieve this either with the NoMAD tool or built-in SSO extension (since macOS 10.15 Catalina). Both options have their advantages over the other.

Other Apple services

For Logicworks, I deployed Content Caching service, which provides frequently downloaded content from the Apple CDN locally. At first glance, it may seem that there is nothing to do, just turning on one switch. However, in a more complex network, Content Caching requires a little more care to set up.

Until the pandemic came, the company ran a successful iPad rental business. For easy re-deployment, I prepared an iMac with Apple Configurator 2. It had been used to erase, update and connect the iPads to MDM via Automated Device Enrollment.

Linux infrastructure

I inherited the old infrastructure for hosting websites and emails using OS X Server. It had to be replaced by something newer. We eventually moved mailhosting to the cloud, but we kept webhosting. I continued the previous cooperation with Vojta Myslivec, who played a key role in putting the web hosting and mail server into operation.

Config Management

The vast majority of the changes in our infrastructure are done using the Ansible config management system. I am a supporter of the philosophy of infrastructure as code and we try to keep the number of ad-hoc actions to a minimum. The code is versioned using git and discussed in the internal GitLab.

VPN

One of our latest projects aims to create our own IKEv2 IPSec infrastructure based on Strongswan project. We use IPsec tunnels to connect corporate networks with the networks of some of our customers. Gateways establish encrypted connections dynamically as needed. Automatic failover to the backup server is also implemented in the event of a failure or maintenance.

We can now replace the obsolete OpenVPN with a significantly better IKEv2 IPSec VPN solution. VPN is designed to be used in split-tunnel + split-dns mode. Deploying the client configuration using the MDM gives us even more options such as making the connections automatic (on-demand).

Monitoring

To monitor our new rapidly evolving infrastructure, I deployed the Icinga 2 monitoring system. At first we used the single-node mode, in which Icinga connects to the monitored systems via SSH. Later, the system was upgraded to a distributed model, where the Icinga 2 service runs on each node and communication takes place in the three-layers: master <-> satellite <-> client.

The states of monitored nodes and services can be administered in the Icinga Web 2 interface. Performance data are stored in the influxDB database, from where Grafana loads them into dashboards.

The second instance of the monitoring system was later deployed to monitor networks our clients. The server in the satellite role checks the status of all network elements and servers mostly using the SNMPv3 protocol. For monitoring purposes, I maintain my own set of monitoring-scripts-logicworks scripts.

DNS

In local networks, we use the DNS server BIND 9 in the functions of an authoritative server providing local zones and a resolver for clients. Zones are automatically replicated to the backup server, and in some cases an ACL is applied to them.

Applications

We have several web applications deployed for practical use. Some are mentioned in other sections because they are closely related to some system. The more independent ones include:

  • NetBox – network documentation from the point of view of interconnection of elements and addressing. We deployed PostreSQL and Redis databases for the application. We document internal and customer networks.
  • Snipe-IT – inventory of company equipment. We also offer it as a service to customers under the name Sensei Inventory.
  • GitLab – collaboration over non-public git repositories.
  • Wiki.js – for documentation. Now in the testing phase. For more usable search capabilities, I deployed Elasticsearch.

Identity

Centralized account management and authentication is provided by OpenLDAP in HA mode. We use the FusionDirectory web application for easier management of the LDAP database.

We have all internal applications connected to LDAP, as well as some cloud applications. The plan is to deploy a SAML endpoint to provide a login gateway to cloud applications and reduce the number of corporate user accounts to a minimum.

Backup

Data must be backed up. I started with an older server, an external disk array, and rsnapshot. But with the need to back up customer data, the disk array turned out to be insufficient.

For the second version of the backup server, we chose Linux RAID6, the btrfs file system and its snapshots. Vojta Myslivec took care of the implementation. We even fount one one nasty bug and reported it.

Webhosting

We offer webhosting in the mode of shared hosting or managed virtual private server. It is all about classic Apache web server, but we support modern technologies such as HTTP/2 and TLS 1.3. Automated deployment of Let’s Encrypt certificates is provided as part of the service.

We run PHP under PHP-FPM, which allows us to separate running processes for individual users and scale the load as needed. We support versions 5.6 to 8.0, but 5.6 is going to be removed soon.

The file system of shared web hosting is limited by quotas for users. Access rights are based on more advanced ACLs, and PHP code certainly cannot modify any files without explicit permission. Sending emails is limited by a rate limit and can be completely disabled if not needed. We provide remote access to files only by secure protocols SFTP and FTPS. Access to the database is possible via Adminer and on request also with phpMyAdmin.

Mailserver

For several years, I helped to manage OS X Server with a classic set of mail components: postfix, dovecot, amavis, SpamAssassin, ClamAV and Roundcube.

The original plan was to build a similar combination on Linux, but it turned out that the eternal fight against spam and mail delivery is something we rather leave to others. We use our mail servers mostly for internal needs.

Virtualization

Our favorite Linux is Debian 10 Buster. It usually runs under Linux KVM virtualization. To a lesser extent, we also use LXC containers.

For more efficient use of hardware resources, we use VMware ESXi running on Apple hardware. This allows us to easily virtualize macOS.

Server hardware

We manage multiple physical servers. They are all equipped with fast SSDs and plenty of RAM. We prefer software RAID in various variants of MD raid according to storage needs.

Other projects

Office network

I designed and implemented a local network for the new office space. Ethernet is secured with 802.1X and Wi-Fi with WPA-2 Enterprise. Users authenticate against the local RADIUS server and then the switch or AP assigns them to the corresponding VLAN. Of course, MDM provides the network configuration to the clients.

Freshdesk ticket PDF export

There was a need to export service sheets in PDF from tickets in our Freshdesk ticketing. In JavaScript, I programmed the Freshdesk application, which uses the Freshdesk API to find out the necessary ticket data and then generate a PDF using the jsPDF library.

NetBoot server

Unfortunately, you will no longer find a network boot on a modern Mac. In 2016, however, it was still the current technology. I prepared a NetBoot server our technicians could use to start Macs over the network and

  • Install all supported versions of macOS (NetInstall)
  • Run applications in a full system environment (NetBoot)
  • Run one of the many Apple diagnostic systems (ASD, MRI).
  • Perform a disk image recovery in a DeployStudio network environment.