MythTV – Java Frontend (JMTV)

As a side project I am thinking of creating java Mythtv frontend. The Mythtv frontend is more that adequate, however I use Nordvpn and Meshnet to access the backend when I am not at home. The Mythtv frontend uses a lot of calls to mysql server which means the frontend pauses/ stalls and is generally unusable. The webfrontend, although useful for configuring the server is not very good for media playback, finding programs etc. So using the Mythtv api allows me to access the backend without the mysql interference. In addition to this the project would have the following goals:

  • Browse, play and delete recordings
  • Schedule recordings
  • Integrate streaming services e.g netflix using a suitable web browser e.g Firefox that handles the DRM issues.
  • Launches games
  • Plays music from either Mythtv or other services e.g youtube music.

As I have not programmed for some time I am going to have to relearn some of the lessons from the previous years.

I do know I will use the WhitestarMediaLibrary project:

This will act as the backend to handle the various libraries be it the Mythtv backend.

BTcreative

Project goal

I had a raspberry pi zero from a failed back up server project and an old usb sound card. This card is a creative 24 live 5.1 surround sound blaster. I decided I would set up a bluetooth receiver to feed audio into my old 5.1 surround sound system. I also like to listen to music when I am gaming. I don’t do this very often but I would like to be able to control the music from my phone including its volume. Sending it via bluetooth into the audio in (the spdif in this case) on the PC seems like a flexible way of doing this. So I thought I would put the two together and get a little more use out of the sound card. So the receiver should:

  • Accept audio from my phone at the highest quality possible.
  • Push audio via the analog out to amp and my PC (via an audio splitter)

Hardware

Software

  • Ubuntu Server
  • Pipewire

Why Ubuntu Server and Pipewire

There are plenty of guides out there for bluetooth speakers, audio recievers which use Raspbian. The issue with most of these is that they don’t appear to use the HD audio. The ones that do either require you to create an account such as with Balena . I didnt particularly didn’t want to do that I wanted to see if I could set up a high quality audio myself. Further investigation I found plenty of guides for bluetooth speakers however I could not see they supported HD audio just pushing the audio through pulse. Although one of the guides was useful for managing the bluetooth pairing.

I eventually found a site that recommended pipewire and pipeplumber which supports HD audio out of the box. This is the method I settled on. However Wireplumber is not easily accessible from Raspbian. It requires a back port and I could not get it to work. So I tried the Ubuntu server and these libraries were available. (although It created additional steps later)

The sites I used can be found at the end this article.

Install

I installed Ubuntu server from using the Raspberry Pi Imager with SSH, wifi login etc. Then I updated and upgraded Ubuntu:

sudo apt update && sudo apt upgrade

From reference 1 in the resources section I installed the pipewire and wireplumber

sudo apt install pipewire wireplumber libspa-0.2-bluetooth

Now I did follow the rest of the guide to get pairing up and running but it didn’t seem to work. so I installed bluetooth

sudo apt install bluetooth

Then I ran bluetooth (type bluetoothctl) from the command line select pair from my bluetooth menu on my phone requested paring and then in the terminal accepted the paring when prompted. I then entered trusted into the terminal and then exited bluetooth by typing exit in the terminal. Trusted means no passcode required next time you pair the phone to the pi.

I tested the audio by plugging in a headphone and played some music on my phone. This worked however the volume was low. Some searches found the archwiki page for Pipewire which had the solution . I had to install the ASLA utilities to get the alsa mixer up so I could increase the volume and save it. This was done by:

sudo apt-get install alsa-utils

Then open the mixer by typing alsamixer. Then amend the volume and type

sudo alsactl store

This will store the volume after reboot. Or it should have done. The volume kept resetting so I had to do follow the guidance here.

Unattended updates

Now I won’t be logging in often in over SSH. If only to pair other devices. So I set unattended updates:

sudo apt install unattended-upgrades
sudo apt install apt-config-auto-update

Test the system will update:

sudo unattended-upgrades –dry-run –debug

If this works okay enable the service (As this is Systemd):

sudo systemctl start unattended-upgrades

This will allow updates without me specifically having to login and run them.

Autologin

I followed this guide to allow autologin otherwise the bluetooth will not connect up:

Autologin no GUI or headless – ubuntu command line

Revelations Digital outputs

I have been running this set up for the last 3 months. What has been the biggest surprise was the sound quality. Now for years I have always read if you can use a digital output you should it is always better. However this setup using the analog outputs to my Denon receiver has much better sound quality than when I have used the Denon’s optical input. I am not an expert in this area but I know what I like. So why is this the case. It could be that optical or spdif output is outdated but I think it is more than this. Again I am not an expert but I can only speculate. So I think what is happening is the DAC on the creative sound card is much better that the DAC in the Denon receiver. I tried the same with the PC connecting the PC’s Asus card to the Denons external 7.1 inputs the result was again improved quality. Dialog in TV was clearer, I hear notes on music I didn’t know were there.

So I guess the lesson is to not always take everything at face value.

Resources

  1. Using a raspberry pi as a bluetooth speaker with pipewire and wireplumber
  2. Autologin no GUI or headless – ubuntu command line
  3. Archwiki page for pipewire
  4. https://dev.to/luisabianca/fix-alsactl-store-that-does-not-save-alsamixer-settings-130i#:~:text=Open%20a%20terminal%20and%20run%20alsamixer%20to%20verify,store%20Make%20the%20script%20executable%3A%20chmod%20%2Bx%20alsa-save.sh
  5. https://ostechnix.com/ubuntu-automatic-login/

What the hell happened?

I may ask myself… what the hell happend? Well COVID meant work took over my life because of covering for shielding staff meant I was working 50 hour weeks. Then berevements have curtailed any ambition of changing carears.

I had to decide what do with this site? I have published the unifinished post I started back then. Moving forward I still like to tinker so any gagetty, ITee type stuff will be posted here. For self documentation if anything else. On my todo list is:

  • Bluetooth Audio receiver using a PI zero W and Creative usb sound card.
  • Update Mythtv server with extra storage
  • Consolidation of PC/ media PC

Oh well has Churchill used to say… keep buggering on.

KUP Assessments – Bash scripts to test endpoints

For integration testing and the testing of endpoints I have previously indicated I am going to use a bash script of curl commands to achieve this. The first iteration of the script is given below:

#!/bin/sh

#URI's
baseURI="http://localhost:8080/kupassessments/"
create="account/create/"

#test the index url
responseIndexPage=$(curl --write-out "%{http_code}\n" --silent --output /dev/null $baseURI) 2>/dev/null
#test the creation of an account
responseAccountCreation=$(curl --write-out "%{http_code}\n" --silent --data "username=Boot&email=boot@r-a-w.org" --output /dev/null $baseURI$create) 2>/dev/null
echo Bash Script Test of KUPassessments
#result of testing the index page
echo Index Page response: $responseIndexPage
echo Create AccountResponse: $responseAccountCreation

The resulting output is:

Bash Script Test of KUPassessments
Index Page response: 200
Create AccountResponse: 404

The 404 is because the query does not contain any data. I also will have run a an SQL script to reset the database.

The database is cleared using the following script ClearAccountTestingData.sql

delete from accounts where username='boot';

This needs to be run each time the bash script starts to ensure that the database is in a state that is expected for the tests. However this needs the location of the script to be included. So it may be best to execute the the SQL from the testing bash script.

Now I have noticed that the second error is not, in fact due to the data already existing but the fact that the request is incorrect. From searching round the request needs to be rewritten so it sends the request in json.

resources:

KUP Assessment Implementation Diary – Connecting up and dependency injection

Here is a question. when do you inject. Tools like Grice do this for you. But if you are to do it manually, when do you do it? how far do you inject.

We could use the KUP assessments class as an injector class however this would break the single responsibility principle so it would be better to have a separate injector class (KupInjector). Then I simply ask it for various services leaving the configuration of the services to IkupInjector. The classes to be injected shall have a mix of setter and interface injection. This reflects the use of both classes and interfaces used in the structure of the system.

My first implementation of the IkupInjector shall use a properties file to retieve the string class names of the classes to be created. This is then placed in the base directory of the server to which can be edited as needed and the service restarted. The first test in the bash script has been created to test the URI for the index page and and account creation. Using curl is something I am definitely need to work on as it took me a significant amount of time to do even the simplest command.

Sources:

KUP Assessments – Implementation Diary -TDD & Refactoring

The next few posts are going to be more like diary entries than posts on specific topics. They will have show my train of thought and therefore will be full of assumptions.

Last year I did a course of TDD with Jpassion.com. It was a useful course and allowed me to also refine and expand on many of my projects. However it did not cover when starting projects you start creating tests. My first instinct was to start straight away however it meant an a lot of mocking and from my searches of the net, there are mixed feelings regarding mocking.

So I have settled on a actual sql database for building the account database as I can also be assured that the SQL statements work. I have had a quandary of refactoring. Consider the following statements:

public Boolean createAccount(String username, String email) throws StoringAccountException, AccountAlreadyExistsException{
connect();
if (checkAccountAlreadyExists(username)) {
close();
throw new AccountAlreadyExistsException();
}
try {
conn.prepareStatement(generateNewAccountSQLStatement(username, email)).execute();
close();
return true;
} catch (SQLException e) {
e.printStackTrace();
close();
throw new StoringAccountException();
}
}


public boolean updateAccountEmail(String username, String updatedEmail) {
 connect();
     if(!checkAccountAlreadyExists(username)) {
      close();
      return false;
     } 
     try {
 conn.prepareStatement(generateUpdateAccountStatement(username, updatedEmail)).execute();
 } catch (SQLException e) {
 // TODO Auto-generated catch block
 e.printStackTrace();
 close();
 return false;
 }
     close();
     return true;
 }
public boolean deleteAccount(String username) {
 connect();
      if(!checkAccountAlreadyExists(username)) {
       close();
 return false;
 }
      try {
   conn.prepareStatement(generateDeleteAccountStatement(username)).execute();
   return true;
   } catch (SQLException e) {
   // TODO Auto-generated catch block
   e.printStackTrace();
   close();
   return false;
   }
 
      
 }

Now I could refactor these as there is a some repetition here. However, some of the methods throw specific exceptions. How do I refactor and keep the exceptions? Or is the issue that some of methods throw exceptions and some don’t. A lack consistency in the interface. Should all the methods throw an exception which may make it easier to refactor? These crud methods however may change further and refactoring could complicate adding the functionality. I will have to return to this issue later and see if I can refactor once other functionality e.g SQL injection prevention is included. I can of course use a finally clause to tidy up the code.

KUP Assessments – An ‘Assessment Creator’ Creates an Account

This requirement relates to the following classes with associated test cases:

  • KupGatehouse
  • Authentication Manager
  • KupFacade
  • KupAssessments
  • AccountManager
  • AccountDatabase
  • HTTP_HeaderChecker

Additionally there is a bash script to test from the frontend of a working system.

Changing Names to Suit Interfaces.

A number of interfaces have altered their names, slightly to reflect that they are interfaces. e.g AccountDatabase has become IaccountDatabase.

Database choice – MySQL

The first implementation of the IaccountsDatabase will be an MySQL database. this is because of the support that is available for MySQL and for not for speed or ideological reasons. I just want it to work.

Database setup

The initial database, which is not embedded so there is flexibility in where the database is located, is set up using a sql scipt. The login details for the AccoundDatabaseSQL class uses to access the database is kept in a properties file on the server. I do worry over the security issues for this but for lack of finding a better solution and the fact that if someone able to access the file, there will be greater problems that just the accounts database being compromised. This will require revisiting later.

IAccount Class creation

My first thoughts are to use a factory pattern for the creation and submission of accounts. The problem is I would be restricted to what the data structure of the concrete account classes would be. Behavior of course would be malleable, but what if there was a method of a class reveal structure a run time. e.g revealDataStructure() it would be up to the IaccountDatabase to store this in its database. The database would also need to be able to handle the structure. In the case of MySQL database this is completely reasonable. so how do I get the factory to insert the data structure. Well I could instead use a method for Iaccount such as loadDataStructure(Map aMap) the class would then wire up its data structure itself. It could also have getDataStructure() which would return a map which it’s keys and values (Strings) could be iterated through to store its data structure. Or should I be creating a IAccount class at all. It is only going to be flattened into a xml or json structure anyway? No. For simplicity of the interface between components, the Iaccout being passed around would be better.

KUP Assessments – Starting Implementation. Last minute Adjustments

Moving functionality

KUP_Gatehouse – removed functionality from checking http_headers to an interceptor class. The same for VERB checker

Need to move authenticationManager to the gatehouse and away from the accountManager as it a better fit as a filter/ interception class

Changing Class names naming scheme

Renamed some of the classes e.g KUP_Assessements so they fit normal naming conventions e.g KupAssessments

Introducing Interfaces

AccountsDatabase should be an interface to allow different database types to be used. Renamed AccountDatabase to IaccountDatabase.

Refined Class Diagram

Refined Class diagram

KUP Assessments Password Schema

  • Block usage of passwords that fall into the top 1000 used (as per https://en.wikipedia.org/wiki/Wikipedia:10,000_most_common_passwords)
  • Will accept password if it is within the most 10000 but not the most 1000 the response will contain a warning and an option to change the password. – The response will not echo the password.
  • Encourage passphrases. By warning if the password is shorter than 10 characters in length, but will not warn if longer than 15 or the password has 2 or more special characters in it such as {}[]@:~!”£$%^&*()_+=-¬`|\/?><
  • Enforce case changing in the password. Mix of upper and lower case.

KUP Assessments Glossary

●Access Control– A means of restricting access to files, referenced functions, URLs, and data based on the identity of users and/or groups to which they belong.

●Address Space Layout Randomization (ASLR)– A technique to help protect against buffer overflow attacks.

●Application Security– Application-level security focuses on the analysis of components that comprise the application layer of the Open Systems Interconnection Reference Model (OSI Model), rather than focusing on for example the underlying operating system or connected networks.

●Application Security Verification– The technical assessment of an application against the OWASP ASVS.

●Application Security Verification Report– A report that documents the overall results and supporting analysis produced by the verifier for a particular application.

●Authentication– The verification of the claimed identity of an application user.●Automated Verification– The use of automated tools (either dynamic analysis tools, static analysis tools,or both) that use vulnerability signatures to find problems.

●Back Doors– A type of malicious code that allows unauthorized access to an application.

●Blacklist– A list of data or operations that are not permitted, for example a list of characters that are not allowed as input.

●Cascading Style Sheets(CSS) – A style sheet language used for describing the presentation semantics of document written in a markup language, such as HTML.●Certificate Authority(CA) – An entity that issues digital certificates.

●Communication Security– The protection of application data when it is transmitted between application components, between clients and servers, and between external systems and the application.

●Component– a self-contained unit of code, with associated disk and network interfaces that communicates with other components.

●Cross-Site Scripting(XSS) – A security vulnerability typically found in web applications allowing the injection of client-side scripts into content.

●Cryptographic module– Hardware, software, and/or firmware that implements cryptographic algorithms and/or generates cryptographic keys.

●Denial of Service (DoS) Attacks– The flooding of an application with more requests than it can handle.

●Design Verification– The technical assessment of the security architecture of an application.

●Dynamic Verification– The use of automated tools that use vulnerability signatures to find problems during the execution of an application.

●Easter Eggs– A type of malicious code that does not run until a specific user input event occurs.

●External Systems– A server-side application or service that is not part of the application.

●FIPS 140-2– A standard that can be used as the basis for the verification of the design and implementation of cryptographic modules●Globally Unique Identifier(GUID) – a unique reference number used as an identifier in software.OWASP Application Security Verification Standard 3.064

●HyperText Markup Language (HTML)- The main markup language for the creation of web pages and other information displayed in a web browser.

●Hyper Text Transfer Protocol(HTTP) – An application protocol for distributed, collaborative, hypermedia information systems. It is the foundation of data communication for the World Wide Web.

●Input Validation– The canonicalization and validation of un-trusted user input.

●Lightweight Directory Access Protocol (LDAP)– An application protocol for accessing and maintaining distributed directory information services over a network.

●Malicious Code– Code introduced into an application during its development unbeknownst to the application owner, which circumvents the application’s intended security policy. Not the same as malware such as a virus or worm!

●Malware– Executable code that is introduced into an application during runtime without the knowledge of the application user or administrator.

●Open Web Application Security Project(OWASP) – The Open Web Application Security Project (OWASP)is a worldwide free and open community focused on improving the security of application software. Our mission is to make application security “visible,” so that people and organizations can make informed decisions about application security risks. See: http://www.owasp.org/

●Output encoding– The canonicalization and validation of application output to Web browsers and to external systems.

●Personally Identifiable Information(PII) – is information that can be used on its own or with other information to identify, contact, or locate a single person, or to identify an individual in context.

●Positive Validation– See whitelist.

●Security Architecture– An abstraction of an application’s design that identifies and describes where and how security controls are used, and also identifies and describes the location and sensitivity of both user and application data.

●Security Configuration– The runtime configuration of an application that affects how security controls are used.

●Security Control– A function or component that performs a security check (e.g. an access control check)or when called results in a security effect (e.g. generating an audit record).

●SQL Injection (SQLi)– A code injection technique used to attack data driven applications, in which malicious SQL statements are inserted into an entry point.

●Static Verification– The use of automated tools that use vulnerability signatures to find problems in application source code.

●Target of Verification (TOV)– If you are performing application security verification according to the OWASP ASVS requirements, the verification will be of a particular application. This application is called the“Target of Verification” or simply the TOV.

●Threat Modeling- A technique consisting of developing increasingly refined security architectures to identify threat agents, security zones, security controls, and important technical and business assets.

●Transport Layer Security– Cryptographic protocols that provide communication security over the Internet OWASP Application Security Verification Standard 3.065

●URI/URL/URL fragments– A Uniform Resource Identifier is a string of characters used to identify a name or a web resource. A Uniform Resource Locator is often used as a reference to a resource.

●User acceptance testing (UAT)​– Traditionally a test environment that behaves like the production environment where all software testing is performed before going live.

●Verifier- The person or team that is reviewing an application against the OWASP ASVS requirements.

●Whitelist– A list of permitted data or operations, for example a list of characters that are allowed to perform input validation.

●XML– A markup language that defines a set of rules for encoding documents