itramblings

Ramblings from an IT manager and long time developer.

By

AD Domain Join Ubuntu with DNS update

Here are a couple of useful articles to help with this task

By

Configure the NTP Server on Windows Server 2016

Copied from https://www.ceos3c.com/2017/07/06/configure-ntp-server-windows-server-2016/

We will use PowerShell to change the NTP Server and we will validate if it worked afterwards.

Configure the NTP Server on Windows Server 2016

On your Windows Server 2016 hit the Windows Button and type: PowerShell and right-click it and select Run as Administrator

Type the following commands

  • w32tm /config /manualpeerlist:pool.ntp.org /syncfromflags:MANUAL
  • Stop-Service w32time
  • Start-Service w32time

Of course, you can take any NTP Server that you want.

Now verify if the time-server was set correctly on your Server 2016 by typing:

  • w32tm /query /status

You should get a reply like this:

Now we will go ahead and verify if our clients sync properly.

 

Verifying if the Time Server was correctly set on our Clients

On a client computer open a PowerShell with right-click and Run as Administrator….

Type:

  • w32tm /query /status

Check the time-server and type:

  • w32tm /resync
  • w32tm /query /status

Then you are able to see that the time-server actually changed like in the example below.

And that’s it! Easy, right?

Always make sure that you use the same time-server in your network and that all your clients are syncing from it. If you have time differences inside of your Active Directory Domain you will run into major issues.

By

Troubleshooting 502 Errors in ARR

Taken from here: https://docs.microsoft.com/en-us/iis/extensions/troubleshooting-application-request-routing/troubleshooting-502-errors-in-arr

Troubleshooting 502 Errors in ARR

by Richard Marr

Tools Used in this Troubleshooter:

  • IIS Failed Request Tracing
  • Network Monitor
  • Winhttp Tracing

This material is provided for informational purposes only. Microsoft makes no warranties, express or implied.

HTTP 502 – Overview

When working with IIS Application Request Routing (ARR) deployments, one of the errors that you may see is “HTTP 502 – Bad Gateway”. The 502.3 error means that – while acting as a proxy – ARR was unable to complete the request to the upstream server and send a response back to the client. This can happen for multiple reasons – for example: failure to connect to the server, no response from the server, or the server took too long to respond (time out). If you are able to reproduce the error by browsing the web farm from the controller, and detailed errors are enabled on the server, you may see an error similar to the following:

Click to Expand

Figure 1 (Click image to expand)

The root cause of the error will determine the actions you should take to resolve the issue.

502.3 Timeout Errors

The error code in the screenshot above is significant because it contains the return code from WinHTTP, which is what ARR uses to proxy the request and identifies the reason for the failure.

You can decode the error code with a tool like err.exe. In this example, the error code maps to ERROR_WINHTTP_TIMEOUT. You can also find this information in the IIS logs for the associated website on the ARR controller. The following is an excerpt from the IIS log entry for the 502.3 error, with most of the fields trimmed for readability:

sc-status sc-substatus sc-win32-status time-taken
502 3 12002 29889

The win32 status 12002 maps to the same ERROR_WINHTTP_TIMEOUT error reported in the error page.

What exactly timed-out?

We investigate this a bit further by enabling Failed Request Tracing on the IIS server. The first thing we can see in the failed request trace log is where the request was sent to in the ARR_SERVER_ROUTED event. The second item I have highlighted is what you can use to track the request on the target server, the X-ARR-LOG-ID. This will help if you are tracing the target or destination of the HTTP request:

77. ARR_SERVER_ROUTED RoutingReason=”LoadBalancing”, Server=”192.168.0.216″, State=”Active”, TotalRequests=”3″, FailedRequests=”2″, CurrentRequests=”1″, BytesSent=”648″, BytesReceived=”0″, ResponseTime=”15225″ 16:50:21.033
78. GENERAL_SET_REQUEST_HEADER HeaderName=”Max-Forwards”, HeaderValue=”10″, Replace=”true” 16:50:21.033
79. GENERAL_SET_REQUEST_HEADER HeaderName=”X-Forwarded-For”, HeaderValue=”192.168.0.204:49247″, Replace=”true” 16:50:21.033
80. GENERAL_SET_REQUEST_HEADER HeaderName=”X-ARR-SSL”, HeaderValue=””, Replace=”true” 16:50:21.033
81. GENERAL_SET_REQUEST_HEADER HeaderName=”X-ARR-ClientCert”, HeaderValue=””, Replace=”true” 16:50:21.033
82. GENERAL_SET_REQUEST_HEADER HeaderName=”X-ARR-LOG-ID”, HeaderValue=”dbf06c50-adb0-4141-8c04-20bc2f193a61″, Replace=”true” 16:50:21.033
83. GENERAL_SET_REQUEST_HEADER HeaderName=”Connection”, HeaderValue=””, Replace=”true” 16:50:21.033

The following example shows how this might look on the target server\’s Failed Request Tracing logs; you can validate that you have found the correct request by matching up the “X-ARR-LOG_ID” values in both traces.

185. GENERAL_REQUEST_HEADERS Headers=”Connection: Keep-Alive Content-Length: 0 Accept: */* Accept-Encoding: gzip, deflate Accept-Language: en-US Host: test Max-Forwards: 10 User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0) X-Original-URL: /time/ X-Forwarded-For: 192.168.0.204:49247 X-ARR-LOG-ID: dbf06c50-adb0-4141-8c04-20bc2f193a61
345. GENERAL_FLUSH_RESPONSE_END BytesSent=”0″, ErrorCode=”An operation was attempted on a nonexistent network connection. (0x800704cd)” 16:51:06.240

In the above example, we can see that the ARR server disconnected before the HTTP response was sent. The timestamp for GENERAL_FLUSH_RESPONSE_END can be used as a rough guide to find the corresponding entry in the IIS logs on the destination server.

date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username sc-status sc-substatus sc-win32-status time-taken
2011-07-18 16:51:06 92.168.0.216 GET /time/ - 80 - 200 0 64 45208

Note that IIS on the destination server logged an HTTP 200 status code, indicating that the request completed successfully. Also note that the win32 status has changed to 64, which maps to ERROR_NETNAME_DELETED. This generally indicates that the client (ARR being the \’client\’ in this case) had disconnected before the request completed.

What happened?

Only the ARR server is reporting a timeout, so that is where we should look first.

In the IIS log entry from the ARR server, we can see that the time-taken is very close to 30 seconds, but the member server log shows that it took 45 seconds (45208 ms) to send the response. This suggests that ARR is timing the request out, and if we check the proxy timeout in the server farm\’s proxy settings, we will see that it is set to 30 seconds by default.

So in this case we can clearly see that the ARR timeout was shorter than the execution of the request. Therefore, you would want to investigate whether this execution time was normal or whether you would need to look at why the request was taking longer than expected. If this execution time was expected and normal, increasing the ARR timeout should resolve the error.

Other possible reasons for ERROR_WINHTTP_TIMEOUT include:

  • ResolveTimeout: This occurs if name resolution takes longer than the specified timeout period.
  • ConnectTimeout: This occurs if it takes longer than the specified timeout period to connect to the server after the name resolved.
  • SendTimeout: If sending a request takes longer than this time-out value, the send operation is canceled.
  • ReceiveTimeout: If a response takes longer than this time-out value, the request is canceled.

Looking at the first two examples, ResolveTimeout and ConnectTimeout, the troubleshooting methodology outlined above would not work. This is because you would not see any traffic on the target server and therefore would not know the error code. Thus in this case of ResolveTimeout or ConnectTimeout you would want to capture a WinHTTP trace for additional insight. See the WinHTTP/WEBIO Tracing section of this troubleshooter as well as the following blogs for additional examples on troubleshooting and tracing:

502.3 Connection Termination Errors

502.3 errors are also returned when the connection between ARR and the member server is disconnected mid-stream. To test this type of problem, create a simple .aspx page that calls Response.Close(). In the following example there is a directory called “time” which is configured with a simple aspx page as the default document of that directory. When browsing to the directory, ARR will display this error:

Click to Expand

Figure 2 (Click image to expand)

The error 0x80072efe corresponds to ERROR_INTERNET_CONNECTION_ABORTED. The request can be traced to the server that actually processed it using the same steps used earlier in this troubleshooter, with one exception; while Failed Request Tracing on the destination server shows the request was processed on the server, the associated log entry does not appear in the IIS logs. Instead, this request is logged in the HTTPERR log as follows:

HTTP/1.1 GET /time/ - 1 Connection_Dropped DefaultAppPool

The built-in logs on the destination server do not provide any additional information about the problem, so the next step would be to gather a network trace from the ARR server. In the example above, the .aspx page called Response.Close() without returning any data. Viewing this in a network trace would show that a Connection: close HTTP header was coming from the destination server. With this information you could now start an investigation into why the Connection: close header was sent.

The error below is another example of an invalid response from the member server:

Click to Expand

Figure 3 (Click image to expand)

In this example, ARR started to receive data from the client but something went wrong while reading the request entity body. This results in the 0x80072f78 error code being returned. To investigate further, use Network Monitor on the member server to get a network trace of the problem. This particular error example was created by calling Response.Close() in the ASP.net page after sending part of the response and then calling Response.Flush(). If the traffic between the ARR server and the member servers is over SSL, then WinHTTP tracing on Windows Server 2008 or WebIO tracing on Windows Server 2008 R2 may provide additional information. WebIO tracing is described later in this troubleshooter.

502.4 No appropriate server could be found to route the request

The HTTP 502.4 error with an associated error code of 0x00000000 generally indicates that all the members of the farm are either offline, or otherwise unreachable.

Click to Expand

Figure 4 (Click image to expand)

The first step is to verify that the member servers are actually online. To check this, go to the “servers” node under the farm in the IIS Manager.

Click to Expand

Figure 5 (Click image to expand)

Servers that are offline can be brought back online by right-clicking on the server name and choosing “Add to Load Balancing”. If you cannot bring the servers back online, verify the member servers are reachable from the ARR server. The “trace Messages” pane on the “servers” page may also provide some clues about the problem. If you are using Web Farm Framework (WFF) 2.0, you may receive this error if the application pool restarts. You will need to restart the Web Farm Service to recover.

WinHTTP/WebIO Tracing

Usually, Network Monitor will provide you with the information you need to identify exactly what is timing out, however there are times (such as when the traffic is SSL encrypted) that you will need to try a different approach. On Windows 7 and Windows Server 2008R2 you can enable WinHTTP tracing using the netsh tool by running the following command from an administrative command prompt:

netsh trace start scenario=internetclient capture=yes persistent=no level=verbose tracefile=c:\temp\net.etl

Then, reproduce the problem. Once the problem is reproduced, stop the tracing by running the following command from the command prompt:

netsh trace stop

The stop command will take a few seconds to finish. When it is done, you will find a net.etl file and a net.cab file in C:\temp. The .cab file contains event logs and additional data that may prove helpful in analyzing the .etl file.

To analyze the log, open it in Netmon 3.4 or later. Make sure you have set up your parser profile as described here. Scroll through the trace until you find the w3wp.exe instance where ARR is running by correlating with the “UT process name” column. Right click on w3wp and choose “Add UT Process name to display filter”. This will set the display filter similar to:

 UTProcessName == "w3wp.exe (1432)

You can further filter the results by changing it to the following:

UTProcessName == "w3wp.exe ()" AND ProtocolName == "WINHTTP_MicrosoftWindowsWinHttp"
You will need to scroll through the output until you find the timeout error. In the example below, a request timed out because it took more than 30 seconds (ARR\'s default timeout) to run.
336 2:32:22 PM 7/22/2011 32.6380453 w3wp.exe (1432) WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:32:23.123 ::sys-recver starts in _INIT state
337 2:32:22 PM 7/22/2011 32.6380489 w3wp.exe (1432) WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:32:23.123 ::current thread is not impersonating
340 2:32:22 PM 7/22/2011 32.6380584 w3wp.exe (1432) WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:32:23.123 ::sys-recver processing WebReceiveHttpResponse completion (error-cdoe = ? (0x5b4), overlapped = 003728F0))
341 2:32:22 PM 7/22/2011 32.6380606 w3wp.exe (1432) WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:32:23.123 ::sys-recver failed to receive headers; error = ? (1460)
342 2:32:22 PM 7/22/2011 32.6380800 w3wp.exe (1432) WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:32:23.123 ::ERROR_WINHTTP_FROM_WIN32 mapped (?) 1460 to (ERROR_WINHTTP_TIMEOUT) 12002
343 2:32:22 PM 7/22/2011 32.6380829 w3wp.exe (1432) WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:32:23.123 ::sys-recver returning ERROR_WINHTTP_TIMEOUT (12002) from RecvResponse()
344 2:32:22 PM 7/22/2011 32.6380862 w3wp.exe (1432) WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:32:23.123 ::sys-req completes recv-headers inline (sync); error = ERROR_WINHTTP_TIMEOUT (12002)

In this next example, the content server was completely offline:

42 2:26:39 PM 7/22/2011 18.9279133 WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:26:39.704 ::WinHttpReceiveResponse(0x11d23d0, 0x0) {WINHTTP_MicrosoftWindowsWinHttp:4, NetEvent:3}
43 2:26:39 PM 7/22/2011 18.9279633 WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:26:39.704 ::sys-recver starts in _INIT state {WINHTTP_MicrosoftWindowsWinHttp:4, NetEvent:3}
44 2:26:39 PM 7/22/2011 18.9280469 WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:26:39.704 ::current thread is not impersonating {WINHTTP_MicrosoftWindowsWinHttp:4, NetEvent:3}
45 2:26:39 PM 7/22/2011 18.9280776 WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:26:39.704 ::sys-recver processing WebReceiveHttpResponse completion (error-cdoe = WSAETIMEDOUT (0x274c), overlapped = 003728F0)) {WINHTTP_MicrosoftWindowsWinHttp:4, NetEvent:3}
46 2:26:39 PM 7/22/2011 18.9280802 WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:26:39.704 ::sys-recver failed to receive headers; error = WSAETIMEDOUT (10060) {WINHTTP_MicrosoftWindowsWinHttp:4, NetEvent:3}
47 2:26:39 PM 7/22/2011 18.9280926 WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:26:39.704 ::ERROR_WINHTTP_FROM_WIN32 mapped (WSAETIMEDOUT) 10060 to (ERROR_WINHTTP_TIMEOUT) 12002 {WINHTTP_MicrosoftWindowsWinHttp:4, NetEvent:3}
48 2:26:39 PM 7/22/2011 18.9280955 WINHTTP_MicrosoftWindowsWinHttp WINHTTP_MicrosoftWindowsWinHttp:12:26:39.704 ::sys-recver returning ERROR_WINHTTP_TIMEOUT (12002) from RecvResponse() {WINHTTP_MicrosoftWindowsWinHttp:4, NetEvent:3}

Other Resources

By

Setting up ASSP, Postfix with SMTP auth for a remote server

By

Exporting Hyper-V Client IP Address Details to a CSV

Took a file to figure out the correct syntax — the following line will export all of the primary IP Addresses along with the name and mac address in a single PS command

Get-VM | ?{$_.ReplicationMode -ne “Replica”} | Select -ExpandProperty NetworkAdapters | ? IPAddresses -ne $null | % { [PSCustomObject]@{ Name=$_.VMName;IPAddress=$_.IPAddresses[0];MacAddress=$_.MacAddress; } }

 

By

The search engine for … Iot, Buildings, Security, etc

Check out https://www.shodan.io/

By

Usefull Docker Scripts

Here are a couple of user docker powershell utility scripts

ConvertFrom-Docker.psm1 – this will take the output from something like

docker ps

and convert it to something like this

docker ps | ConvertFrom-Docker

ContainerId : fbf1955f00a9
Image       : portainer/portainer
Created     : About an hour ago
Status      : Up About an hour
Ports       : 0.0.0.0:9000->9000/tcp
Name        : portainer

Here is the code

function PascalName($name){
    $parts = $name.Split(" ")
    for($i = 0 ; $i -lt $parts.Length ; $i++){
        $parts[$i] = [char]::ToUpper($parts[$i][0]) + $parts[$i].SubString(1).ToLower();
    }
    $parts -join ""
}
function GetHeaderBreak($headerRow, $startPoint=0){
    $i = $startPoint
    while( $i + 1  -lt $headerRow.Length)
    {
        if ($headerRow[$i] -eq ' ' -and $headerRow[$i+1] -eq ' '){
            return $i
            break
        }
        $i += 1
    }
    return -1
}
function GetHeaderNonBreak($headerRow, $startPoint=0){
    $i = $startPoint
    while( $i + 1  -lt $headerRow.Length)
    {
        if ($headerRow[$i] -ne ' '){
            return $i
            break
        }
        $i += 1
    }
    return -1
}
function GetColumnInfo($headerRow){
    $lastIndex = 0
    $i = 0
    while ($i -lt $headerRow.Length){
        $i = GetHeaderBreak $headerRow $lastIndex
        if ($i -lt 0){
            $name = $headerRow.Substring($lastIndex)
            New-Object PSObject -Property @{ HeaderName = $name; Name = PascalName $name; Start=$lastIndex; End=-1}
            break
        } else {
            $name = $headerRow.Substring($lastIndex, $i-$lastIndex)
            $temp = $lastIndex
            $lastIndex = GetHeaderNonBreak $headerRow $i
            New-Object PSObject -Property @{ HeaderName = $name; Name = PascalName $name; Start=$temp; End=$lastIndex}
       }
    }
}
function ParseRow($row, $columnInfo) {
    $values = @{}
    $columnInfo | ForEach-Object {
        if ($_.End -lt 0) {
            $len = $row.Length - $_.Start
        } else {
            $len = $_.End - $_.Start
        }
        $values[$_.Name] = $row.SubString($_.Start, $len).Trim()
    }
    New-Object PSObject -Property $values
}

function ConvertFrom-Docker(){
    begin{
        $positions = $null;
    }
    process {
        if($positions -eq $null) {
            # header row => determine column positions
            $positions  = GetColumnInfo -headerRow $_  #-propertyNames $propertyNames
        } else {
            # data row => output!
            ParseRow -row $_ -columnInfo $positions
        }
    }
    end {
    }
}
  
Export-ModuleMember -Function * -Alias *
# e.g. :
# docker --tls ps -a --no-trunc | ConvertFrom-Docker | ft

.\Docker-Utils.psm1 – this is a set of utility functions and aliases that make life a littler easier.

docke-ps-ext # retrieve docket ps info with IP address

ContainerId : fbf1955f00a9
Image : portainer/portainer
Created : About an hour ago
Status : Up About an hour
Ports : 0.0.0.0:9000->9000/tcp
Name : portainerIPAddress : 172.22.232.8

Here is the code

function Docket-Get-IP() { param ([string] $containerId) return docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $containerId }

function Docker-List-IPAddress { 
  docker ps | ConvertFrom-Docker | % { 	
    $dc = New-Object PSObject -Property @{ ContainerId=""; Image=""; Created=""; Status=""; Ports=""; Name=""; IPAddress="" }
    
    $dc.ContainerId = $_.ContainerId
    $dc.Image = $_.Image
    $dc.Created = $_.Created
    $dc.Status = $_.Status
    $dc.Ports = $_.Ports
    $dc.Name = $_.Names
    $dc.IPAddress = (Docket-Get-IP($_.ContainerId))
    
    $dc | Select ContainerId, Image, Created, Status, Ports, Name, IPAddress
  }
}

Remove-Item docker-ips -ErrorAction SilentlyContinue
Set-Alias -Scope Global -Name docker-ps-ext -Value Docker-List-IPAddress

 

By

Quick computer setup using Chocolatey

Here is a quick script for setting up a new computer using Chocolatey

@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

cinst -y 7zip
cinst -y notepadplusplus
cinst -y notepadreplacer
cinst -y javaruntime
cinst -y spacesniffer
cinst -y sysinternals
cinst -y firefox
cinst -y googlechrome
cinst -y vcredist2010 
cinst -y vcredist2012 
cinst -y vcredist2015 
cinst -y silverlight
cinst -y mremoteng

 

By

Ramblings on blockchain

Some thoughts on what could be done with blockchain

  • Supply Chain/Inventory Validation
    • Blockchain opens up the possibility of building a “trusted supply chain” that is not just dependent on partners reporting in a reactive manner.  Through the use of blockchain it is now possible to validate the who, what, when, where, and how many in a way that cannot be forged.  Consider the idea of recording entry/exit of a “group of items” along with their unique identifiers in a blockchain ledger.  The idea being that now here is an immutable record that traces things down to the item level that can be validated at any time along the way without relying on specific partners data systems.  This could have a huge image on fraud, product loss, “accidental redirection” of inventory, etc.
  • Embedded Software Validation
    • Consider the recent IOT attach on DynDns that took out DNS on the east coast for a few hours.  This could have been prevented if there was a way for embedded systems to not only validate that individual components (dlls, code, etc) are valid but also that the integrity of the entire system state is in a valid state (not compromised).  This could be done by setting up a blockchain that stores enough information for an embedded system to validate that all components are a state that the provide considers “valid” and “update to date”.
  • Intellectual Collaboration
    • Today more and more companies are working together to bring new products to market and are seeing a lot of value from these partnerships.  The hard part is keeping track of who contributed what and who had the first real substantive idea that lead to a product/solution.  Blockchain could be used to setup an “ideation and collaboration” ledger system that would in turn establish a “trusted system of record” for the entities involved in the collaboration.
      Note: This could just as easily be used internally.
  • Licensing/Royalties
    • Blockchain opens up the potential for a self-enforcing contract that can be validated real-time in an automated way.  Consider the music industry (or any industry that licensing something for that matter).  Today these contracts are managed by intermediaries, manually maintained, and “audited” for compliance and payment.  With blockchain you have the ability to not only record a “purchase”, you can also record the “terms” associated with the purchase.  This is then something that can be validated at any point in time allowing systems to be designed to easily and cost effectively report usage (and potentially track incorrect usage through thumbprints).

At the end of the day most of these ideas come down to the idea of leveraging blockchain for managing “contracts” in a way that can be automated and validated in real-time and in a standard and cost effective manner.

By

Freenas and shell commands for restarting AD connection

Here is a quick set of shell commands to reset and enable AD on Freenas 9.x

service ix-kerberos stop
service ix-nsswitch stop
service ix-kinit stop
service ix-activedirectory stop
service ix-pam stop
service ix-cache stop

sqlite3 /data/freenas-v1.db "update directoryservice_activedirectory set ad_enable=1;"
echo $?
service ix-kerberos start
service ix-nsswitch start
service ix-kinit start
service ix-kinit status
echo $?
klist

python /usr/local/www/freenasUI/middleware/notifier.py start cifs
service ix-activedirectory start
service ix-activedirectory status
echo $?
python /usr/local/www/freenasUI/middleware/notifier.py restart cifs
service ix-pam start
service ix-cache start &