HowTo

Archived Posts from this Category

2019-08-20

Cross-Organization Calendar Sharing with O365

Posted by Robert Auch under Exchange, HowTo, Personal, Technology | Tags: , , , |
Leave a Comment 

Or: letting my wife see my personal calendar.

So yes, I keep all my personal calendar items in the totalnetsolutions.net O365-hosted account calendar. But that doesn’t stop our family from having the calendar sharing problems that apps like GetClockwise https://www.getclockwise.com/ Cozi https://www.cozi.com/ or even shared Google or iCloud calendars try to solve.

The problems we ran into with Cozi were that we had to actually put the items there, then subscribe to the Cozi calendar from everywhere, when most of the time i’m looking at a calendar, I want to look at a single calendar. Can I agree to host this PTO event, or is my wife going to be out of town? Seeing the free/busy from her calendar, my calendar, etc. in one place got difficult.

So I went back to basics: Exchange can publish free/busy data. If everyone is an O365 subscriber, can’t we just share cross-organization? And the answer is that yes we can… IF the organization allows it! Since I am the admin for the TNS organization, I can share my calendar out, and not have to forward events everywhere. here’s how:

First, the admin has to follow the steps here: https://docs.microsoft.com/en-us/office365/admin/manage/share-calendars-with-external-users?view=o365-worldwide – Log into the Admin Center, click “Show all”, then “Settings” and finally “Serivces & add-ins”. The list of add-ins will change (as of today it includes Azure MFA, Calendar, Cortana, Directory Synchronization, and a host of other services), but you’re looking for “Calendar”. Click that and you’ll get this pane slide in from the left:

O365 Admin Center Calendar Sharing Settings

Click “Let your users share their calendars with people outside of your organization who have Office365 or Exchange” and “Save Changes”.

BUT WAIT, THERE’S MORE! Now click “Go to the Exchange admin center to manage additional settings”, and click “organization”:

O365 Admin Center Organization Sharing page

From here, under the “individual sharing” section you can create a new policy (which has to be assigned to users), or hit the pen to edit he “Default Sharing Policy” and get this page, where you can add the domain you want, and the amount of data to allow users to share (up to and including):

O365 Sharing Rule for Default Policy

Once that’s shared, you can save everything and log out. Now it’s up to the individual users to follow the settings here:

https://support.office.com/en-us/article/share-your-calendar-in-outlook-on-the-web-7ecef8ae-139c-40d9-bae2-a23977ee58d5#bkmk_beta

But realize that private events will NOT be visible in a shared calendar view. You’ll be able to load the free/busy data when scheduling, but not see the events in an overlay, even if you allow greater organizational sharing (as I have in the screenshot above). Since my wife’s work, and my other work account are both O365 tenants, I was able to share more-relaxed versions of my calendar with both organizations, send to the individual email addresses, and now we can see synchronized calendar overlays in our own work Outlook calendars.

 

2018-11-01

Windows Domain Controller boot failure: Database Corruption

Posted by Robert Auch under Domain Controllers, HowTo, Windows | Tags: , , , , , , |
Leave a Comment 

The UPS on the lab servers is near the end of its life, so we’ve changed the shutdown script in power outages to simply hard-power off some of the servers (using Rob’s fork of ESXIDown from https://github.com/docsmooth/esxidown ). Of course, the second DC had to be one of them, so that there would be enough battery left for the SQL server and FSMO / PDC to shut down cleanly. THis means that the second DC often fails to boot properly when the power comes back after our yearly ComEd outage.

So, how do you fix this without doing the research every time? Thankfully, modern versions of Windows can be configured to boot to recovery mode. If you log in and launch a command prompt, you can run the following commands to find and repair problems:

  1. Launch the command prompt.

    recovery mode cmd.exe window example

    Launching CMD.EXE in recovery window

  2. navigate to your drive with the NTDS.DIT file (on our lab server, it’s always d:\windows\ntds:
    d:
    cd\windows\ntds
  3. Run an ESENTUTL checksum on the Active Directory database file ntds.dit:
    d:\windows\system32\esentutl.exe /k ntds.dit

    ESENTUTL.exe completed checksum

    ESENTUTL.exe completed checksum

  4. Even though that’s successful, you’ll probably fail an integrity check:
    d:\windows\system32\esentutl.exe /g edb
    On our server, that always generates the error:

    The database is not up-to-date. Integrity check may find that this database is corrupt because data from the log files has yet to be placed in the database. It is strongly recommended the database is brought up-to-date before continuing! Do you wish to abort the operation?

  5. If the checksum passed and you get this error, try rebooting into Directory Services Repair Mode – exit the command prompt and hit “Restart” and then pound on F8 to get the DSRM boot option.
  6. If DSRM bluescreens, then you need to go deeper into esentutl.exe to repair your DB. If this is NOT your only DC, you will have data loss, but it should replicate from other DCs back into this one, so it shouldn’t be a huge problem.
    If DSRM works, try an ntdsutil in cmd.exe:
    ntdsutil.exe
    activate instance ntds
    files
    recover
  7. If ntdsutil file recovery errors with

    Could not initialize the Jet engine: Jet Error -543.
    Failed to open DIT for AD DS/LDS instance NTDS. Error -21478418113

    then you need to do more work with esentutl, but can continue inside DSRM, rather than having to reboot.

    If you couldn’t boot into DSRM, continue as below, but from the recovery install CMD.exe. I’ll continue these commands from THAT pathing, since it turns out not bothering to jump into DSRM makes for a faster recovery, with ensured data loss. I don’t care abuot data loss in our lab though.

  8. Try to recover the database: d:\windows\system32\esenutl.exe /ml d:\windows\ntds\edb
  9. If that doesn’t work, delete the edb.log files, then try recovery again. You’ll get an error like

    Operation terminated with error -501 (JET_errLogFileCorrupt, Log file is corrupt) after 2.123 Seconds.

    So backup your logs to a new location or delete them outright:
    mkdir log-backup
    move edb*.log log-backup
    move edb.chk log-backup

  10. Recreate the log files with a hard recovery of the database:
    d:\windows\system32\esentutl.exe /p d:\windows\ntds\ntds.dit
    You’ll get an error saying you should only run this on damaged or corrupt databases. The checks before this have proven that that is the case in this situation, so click “OK”.
  11. This should restore the database and complete successfully. Reboot and test it. If it doesn’t work, or doesn’t boot, try again in all offline mode without jumping to DSRM.
  12. If that still doesn’t work:
  13. Recheck the database integrity:
    cd \windows\ntds
    esentutl /g: ntds.dit
  14. Do a database repair again:d:\windows\system32\esentutl.exe /p ntds.dit
  15. And reboot when that completes successfully. You *should* now boot properly, and the edb.chk and edb.log files should get rebuilt.
 

2017-09-13

Kubuntu 17.04 on Dell e5470 for extreme battery life

Posted by Robert Auch under HowTo, Linux, Mobile, Personal | Tags: , , , , , , |
Leave a Comment 

So apparently it’s been 5 years since I last updated this series: https://www.totalnetsolutions.net/2012/12/09/lenovo-t430-running-kubuntu-12-10-for-extreme-battery-life/

i’ve restarted most of these configurations over the past 5 years, especially as I’ve switched away from WWAN to tethering, and from spinning rust to SSD, but a lot of the core concepts remain: about 6 years ago my battery died after 3.5 hours of VM troubleshooting, while on a flight, and I lost some data in the emergency “go to sleep, not hibernate”, which cost me 2 hours of rework in the hotel at midnight. My goal, now, is “be able to work multiple simultaneous tasks, have a VM running, and still get super-long battery life when I need it, but not impact performance noticeably.”

With powertop reporting <5W (14h remaining) power consumption while idle, and <6W (11 hours) with firefox open while I start to write this post in August, 2017, I think I’ve hit the mark reasonably well.

As with the Lenovo T430 in the previous post, everything I care about works right out of the box, but when I first started my custom kernels, I missed a few things that I had to add back in before writing this up.

Hardware


CPU: Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
Memory: 16GB RAM
VGA compatible controller: Intel Corporation HD Graphics 520 (rev 07)
Ethernet controller: Intel Corporation Ethernet Connection I219-LM (rev 21)
Network controller: Intel Corporation Wireless 8260 (rev 3a)
Audio device: Intel Corporation Sunrise Point-LP HD Audio (rev 21)
Bluetooth device: Intel (integrated on the USB)
Synaptics Touchpad and Twiddler Mouse

Jump to main sections with these links:
CPU Configuration
Network Configuration
Video Configuration
Encryption / Security configuration
Battery saving configuration
Custom kernel .config

CPU and Battery


I have a series of posts on my love of getting the most performance and battery life I can from my systems, see the last and first of the series for a bit more, or dig through my twitter on the subject. What’s new this year is the latest i5 Core CPU with Linux 4.4 has a new “pstate” performance governor that’s not actually buggy anymore, if you configure it right. I used to use the acpi_cpufreq governors “ondemand” on AC and “conservative” on battery. But the new pstate drivers apparently perform better (thanks Phoronix) AND scale down for battery savings better, so I needed to switch that. Since I was switching governors, I figured it was time to re-check my 2007 finding that moving from “GENERIC_CPU” to “MCORE2” saved me 30+ minutes alone.

Well, it does. But I didn’t keep the data, sorry. What this means is I once again needed to custom compile a kernel to get the right CPU options, and to get the new pstate driver. Since I was in doing that, and since the 2012 post, I’ve moved away from the default kernel Scheduler to Colin Kalvis’ BFS scheduler, so we get to patch THAT in as well. More on those options down at the custom kernel config section, but the point here is that BFS added some stability to heavy “running multiple VMs, and processing 4GB of raw data in Perl” swapping problems I was having, even with 16GB RAM, as well as not hurting my battery life, with a high possibility of 10-20 minutes extra life on normal operations.

The battery in the system is designed at 8157000 mWh, and after 6 months is down to 6621000 mWh. The “amount of time running” is based on the past 2 months, not day 1 of receiving the laptop.

Lastly, I’m still using cpufreqd, but the configuration is vastly simplified – pstate “powersave” when my AC is not plugged in, or the battery’s below 70%, and pstate “performance” otherwise. Ubuntu fixed the broken cpufreqd daemon sometime in 2014, so I’m back to the distribution default version of that, yay!

My custom cpufreqd.conf.

Network


I stopped using my jumbo frames script from here in Kubuntu 16.10, because apparently NetworkManager can figure that out on its own, and it’s been relatively successful. My wireless adapter connects to the new Netgear T6400 at near gigabit speed, but the R6400 doesn’t support jumbo frames itself, so I’m segmenting off some new VLANs to break the Jumbo Frames hosts from the wireless / nonintelligent hosts. That’ll mean resurrecting my jumbo frames script to instead set the VLAN Tag when I’m home.h

Sound


Sound has always been a joke for Linux users, but the Intel HD-Audio has been really solid for me for several years, especially with pulseaudio actually being relatively stable for me. When I recieved the laptop, I was having a problem where full-duplex audio was causing what appeared to be a storm of interrupts that hung the entire laptop. But about 2 months of debugging resulted in “I built a new kernel, and now it works fine.” I don’t know if it was a bug in the codec in the kernel, or something that silently patched. How I have Bluetooth audio headset, bluetooth headset for online conference calls, and appropriate switching for apps and reminders (reminders / alerts go to speakers and bluetooth in my config, in case I take the headphones off), with the options int he PulseAudio configuration in KDE. I have no “.asoundrc” or /etc/asound or /etc/pulse or ~/.pulse/client.conf anymore either, which is great!

Video


For the first time in years, I do not have a multi-graphics card system to deal with. The i915 driver works out of the box and is unremarkable, but functional. And great for battery life. But boring to discuss.

Encryption and Security

Encryption

During installation, I chose the option to use an encrypted LVM volume. This uses DM-Crypt to encrypt the full HDD, so that it has to be unlocked at boot time. The Kubuntu installer seems to forget this fact, so it also asks you to set up ecryptfs private home directories, which is NOT neccessary for a single-user laptop, since the whole OS is already encrypted. The only oddity with dm-crypt is that sometimes the splash screen prompt to unlock the computer doesn’t show. Originally, if I just wait for disk activity to disappear, and have a blank screen, I can just type the passphrase, and it’ll still unlock successfully. But I instead made a change to /etc/default/grub:


GRUB_CMDLINE_LINUX_DEFAULT="noquiet nosplash"

And now I don’t get the splash screen, and the prompt comes up properly right away. And I get all the hacker-y looking boot errors from systemd.

Security

Because this laptop has some sensitive work information on it, I wanted to get a bit more paranoid with the “unattended on a conference table” and “connected to a public wifi network” situations, especially since I actually have OpenSSH listening on all interfaces (yes, I ssh into my laptop from my phone more often than you do). I purchased a multi-protocol Yubikey and downloaded and installed the Yubikey PAM module for Challenge-Response, with the instructions on their website, here. Combined with Active Directory authentication, my cached user can only log in if the Yubikey is inserted into the laptop. So when I step away in meetings, the laptop locks, and my password can’t be cracked.

For additional security, I enrolled the root account on my laptop into my Password vault which rotates the root password every 14 days with a 50+-character random passcode, so even an attacker getting physical access once it’s booted (decrypted) will have little chance of breaking into the box even when I have the yubikey in place.

Additional Power Savings


I still run laptop-mode tools to cut down on power utilization from non-CPU peripherals. I could get more by having the ethernet port actually turn off when on battery, but I actually use it on battery quite a lot, so I’m not sure the hassle of re-enabling it is worth the battery savings. Here are the configurations I use:
intel-hda-powersave
intel-sata-powermgmt
intel_pstate
laptop-mode
runtime-pm
wireless-iwl-power
cpuhotplug
bluetooth
battery-level-polling
ethernet

What I’m now getting is 5-7W of power utilization while online with firefox and chrome both running, bluetooth running, and no VMs. Booting my Windows VM in VMware Workstation bumps me up to 15-20W, but I’m still getting 5 hours of battery life with no features disabled AND running a full Windows VM (the Windows VM has battery detection disabled, too). My non-VM battery life is reporting in the 9-11 hour range, but I’ve never had to use it that long to worry.

Kernel Config


I use the Ubuntu Kernel sources, mostly because the laptop tells me when there’s new Kernel sources with security fixes. I’m using BFS as my scheduler, which is fantastic when I get into “3 VMs using 12GB RAM and a reporting job wanting another 6GB” swap death. I have enough keyboard control to kill the reporting job, then shut down the VMs, and try the reporting job again. Before BFS, I either waited 6 hours, or rebooted the whole damn laptop.
BFS patches are here. If that doesn’t make sense, don’t use them. Please.
My custom kernel .config is here.

Building

The build system I use is the same as in 2012:

sudo apt-get install fakeroot build-dep linux-image-`uname -r`
sudo apt-get install linux-source
sudo usermod -a -G src YOUR_USERNAME

Now log out and back in, so that you’re a member of the “src” group.

cd /usr/src
sudo chown -R $USER:src .
tar -jxf ./linux-source-4.4.0/linux-source-4.4.0.tar.bz2
ln -s linux-source-4.4.0 linux
cd linux
wget http://www.totalnetsolutions.net/wp-content/uploads/2012/12/rob-config-20121204c.txt
mv rob-config-20121204c.txt .config
make oldconfig
make menuconfig

Make any changes you want in here, then exit and save

make-kpkg --initrd --rootcmd fakeroot --append-to-version=.20170912a kernel_image kernel_headers

You’ll get 2 DEB files in /usr/src that you can then install and boot to. the “append-to-version” I use as a dating system for my kernels. “20170912a” means the 2nd kernel attempt on September 12, 2017, the day I’m writing this post (first attempts get no letter).

 

2017-08-24

Invalid ISTG in RODC-only Site

Posted by Robert Auch under Domain Controllers, HowTo, Windows
Leave a Comment 

Our primary lab here at totalnetsolutions.net is a bit space-constrained (RAM constantly 50% overcommited in ESX), so we have it limited to only 2 domains, a parent, and a child. There’s 2 sites, one of them is a “BranchOffice” with only an RODC. This serves us well for the majority of customer cases, especially since the DHCP pool serves the BranchOffice site, which is routed via DSL to the CorpOffice site.

Recently we needed a 3rd domain (2nd child) for testing a particular user/group creation scheme for a customer. Upon completing testing in August, to reduce RAM utilization on the ESX host (and because it makes the AD Schema even more “real-world”, we dcpromo’ed the DC for the 3rd domain, deleting the entire domain. We deleted the objects, and ran the ntdsutil metadata cleanup, like good AD admins. Then we watched the logs fill up with Information events only for a few weeks, and forgot about the episode.

The thing was: the DC in the 3rd domain, because it was short-lived, was left with its DHCP address (it was never a DNS server), so it got added to the BranchOffice site. According to this 2009 Technet AD Troubleshooting Blog entry from Ingolfur Strangeland, RODCs won’t register as the Intersite Topology Generator (ISTG) for the site, but they will perform that role for themselves. (Background on the ISTG.) Because of this, the *new* DC, which was writeable, took over the role of the ISTG for the site, and made sure that all the replication connections went through it, as the only writeable DC in the site.

The problem is, we removed this server. When we removed the writeable DC in the BranchOffice site, there was no other server available to write “I’m now the ISTG” and replicate that setting outbound to the other site(s)… because the only remaining server was an RODC.

We discovered the problem 2 months later (Rob had a baby and wasn’t paying any attention to the lab for a while) when, after creating 1,000,000 new users in the lab, replication was surprisingly slow, but only into the Branch office. So slow that when we joined new computers, they’d boot up with “Service Principal Unknown” errors, and AD users couldn’t log in. In Active Directory Sites and Services we saw that the ISTG Server and Site were both “Invalid”.  This post from 2011 discusses how to move this, but not if it works for RODCs… the good news is that it does:

  1. open the Configuration container, like with adsiedit.msc
  2. Expand CN=Sites, and then the site with the broken ISTG (likely the site with the RODC)
  3. Double-click to open the properties of CN=NTDS Settings
  4. Find the value: “InterSiteTopologyGenerator” and paste in the full DN (from the Configuration Container, not the RootDN) of the RODC
    1. This is the “distinguishedName” value of the CN=<servername>,CN=<sitename>,CN=Sites,CN=Configuration object of the server in the site in question that *should* be the ISTG.
  5. Click “OK” to Save, use ‘repadmin’ or dssite.msc (AD Sites and Services) to force replication and wait 15 minutes (or your own inter-site replication time)

 

 

2017-08-24

Signing RPM packages for Multiple RHEL versions

Posted by Robert Auch under HowTo, Linux, Open Source Software, Security | Tags: , , , , , , , , , , |
Leave a Comment 

Signing RPMs is supposed to be easy:

gpg --list-keys
cat > ~/.rpmmacros < <EOL
%_topdir /home/rob/rpmbuild
%debug_package %{nil}
%_signature gpg
%_gpg_name Rob A <me@totalnetsolutions.net>
%{__gpg} \
gpg --digest-algo=sha1 --batch --no-verbose --no-armor \
--force-v3-sigs --passphrase-fd 3 --no-secmem-warning -u "%{_gpg_name}" \
-sbo %{__signature_filename} %{__plaintext_filename}
EOL
rpm --resign

But there are a lot of caveats:

  1. If you run this on RHEL or CentOS or Scientific Linux 6.x and have an RSA, rather than a DSA, GPG key, any older systems (5.x or 4.x) won’t be able to properly decode the signatures. If you have a DSA key (and only v6+), the --force-v3-sigs is enough.
  2. If you run this on RHEL or CentOS 5.x, you need to remove the “–force-v3-sigs” option from the .rpmmacros line
  3. http://technosorcery.net/blog/2010/10/10/pitfalls-with-rpm-and-gpg/ is a bit wrong – RSA keys don’t always work, DSA do (there’s apparently a RHN KB article about this, if you have support and licenses to read it. I don’t right now).
  4. RHEL / CentOS / Scientific Linux 7 won’t accept (without warning) RPMs signed with weak keys (or weak digests, in some cases, like sha1)

So, based on the systems you’re trying to build your package for, ensure the signing key you’re using and the digest algorithm you’re using, are supported across all the versions you expect to support… or build multiple RPMs, and put them in separate repos for each OS version.

 

Next Page »