Uly's Job History

Home

Resume

Job History

Languages

About

REALNETWORKS, INC., Seattle, WA 2000–2016

Senior Systems Engineer (2008–2016)
Senior Systems Administrator (2004–2008)
Systems Administrator (2003–2004)
Associate Systems Administrator (2002–2003)
Junior Systems Administrator, contractor through Comforce (2000–2002)

During my tenure at RealNetworks, my accomplishments included maintaining legacy systems, migrating legacy services to new systems, and designing and deploying new services.

AWS:

To reduce their datacenter footprint, RealNetworks moved many services to AWS. I assisted with this migration. I both moved old services to AWS and set up new services there. I set up VPCs, subnets, and NAT instances. I've brought up EC2 instances, set up ELBs, and installed certs. I pushed files into S3, and set up users, groups, roles, and policies in IAM.

DNS:

RealNetworks had a heterogeneous DNS farm, including Linux BIND, FreeBSD BIND, and BT Diamond appliances. There were three masters, with inconsistent configs across the farm. Sometimes the same record would resolve differently on different servers, and it wasn't always clear which master was the canonical source for a zone. None of the servers were monitored, nor were changes tracked.

I consolidated the zones to one master and implemented identical configs across the farm, generated from a central location. I also moved them all to BIND, running on CentOS Linux. I set up change tracking on the zones and set up monitoring for all of our DNS servers.

All authoritatives were in the RealNetworks headquarters building, which lacked redundancy and made our DNS vulnerable to outage. I moved them to the Westin datacenter, and then moved one to a datacenter in Japan. Now that RealNetworks is dramatically reducing its datacenter footprint, I've moved them into four different AWS regions. All the migrations were performed without an interruption in service.

CVS and SVN

When I took over CVS and SVN, they ran on very old hardware, with outdated versions of CentOS. I migrated them to new servers in the Westin datacenter. The CVS server had its own homegrown system controlling access, so I needed to modify several Python scripts and a C source file to get it working under a modern OS.

nginx Redirection:

Marketing needed the ability to redirect a variety of domain names and URLs. I set up and supported a robust and easy-to-use nginx-based redirection service.

Ecommerce Platform Preprod Support:

I took over maintenance, support, and upgrades of the Ecommerce Platform's preproduction stack. This included several different environments, each running many different applications on separate servers, mostly VMWare VMs. I stabilized some as they were, and upgraded others to newer OS versions. I started a project to migrate all of them to a standard OS and server build, configured by Chef.

Release Automation:

RealNetworks uses a complex homegrown release system consisting of multiple systems running a mix of Perl, PHP, and RHTML. Several different groups rely on it to release code and configs to both preprod and production. I took over support and development on it, performed bug fixes, and upgraded some of the systems.

FTP/SFTP:

I built an FTP/SFTP farm for both internal and external use.

JIRA and Confluence:

JIRA is a ticketing system RealNetworks uses for bug, request, and project tracking. Confluence is used as a documentation repository. I took over support, stabilized the services, documented the setup, and automated maintenance and upgrades for these systems. I set up test instances for development purposes.

PagerDuty:

I wrote a service in Ruby that monitors our JIRA and Confluence servers, sends alerts to PagerDuty if they become unreachable, and automatically resolves incidents when the servers are brought back up.

Logs:

RealNetworks collects many server logs for data analysis purposes. When I became involved, three different servers handled log collection using a patchwork of many scripts and inconsistent methods. I consolidated log collection from both in-house and third party servers. I set up a single location for logs to be pushed to and developed a service which could pull logs from any source. I migrated this service from RN's old location to a new datacenter, and later to AWS S3 buckets.

Streaming Redirector:

RealNetworks operated a large heterogeneous farm of media servers, controlled by a homegrown streaming redirector which was essential for balancing server loads in the face of unpredictable audience sizes. I began supporting this service as part of a team. Over time, I took on increasing responsibility until I became the sole sysadmin and developer.

RackMonkey:

RealNetworks uses a program called RackMonkey to manage its datacenter. This service had not been well maintained, and was at risk of failure just as RN began to relocate - a time when tracking physical assets became critical. I rescued the service, set up backups, and set up automated daily reporting to show the migration of hardware. I set up an automated job to generate a PDF report, breaking out rack usage by department, allowing the Data Center Management group to accurately assign internal charges.

CUCINA HOLDINGS, INC. (Caravali Coffees), Kent, WA 1996–2000

I was hired on at Caravali to be a roaster lead. I moved into quality assurance, for all aspects – roasted coffee, green coffee, and packaging integrity.

During my last 18 months there, I also took over support of the LAN, which had a Novell fileserver, a bunch of win95/win98 systems, and a link down to the corporate system in Sacramento.

I was laid off when the plant shut down, I could have moved to Sacramento but chose not to.

STARBUCKS COFFEE, Seattle, WA 1992–1996

I started at Starbucks in production, and then moved to Machine Operator. I became production lead when we opened a new plant in Kent, and was highly involved in setting up the machines and processes there.

After that, I trained as a roaster, and after being a roaster for awhile, became a roaster lead.

I left to be a roaster lead at Caravali.