- User Story Testing
- Issues and bugs caught during testing
- Status Code Testing
- Functionality Testing
- Security Testing
- Browser Testing
- Responsivity Testing
- Code Validators
- Performance and Web Development Tools Testing
click for tests
-
Easily understand the purpose of this web application. - PASS
- The landing page is almost wholly dedicated to elucidating the purpose and functionality of the application.
- The application's name encapsulates meaning, giving the first clue towards purpose.
- Then the landing page summarises the idea with a pithy tagline.
- Then it expands that a tad more with icons and more specific language.
- The timeline further explains purpose with the temporal component outlined.
- The next section is aptly titled "How does this work"? and it further expands on the competitive aspect.
- Finally, there is a collapsible that clarifies any and all questions a user might still have at this stage.
-
Quickly and easily understand how to navigate and use the application. - PASS
- As abovementioned the purpose and rules are evident.
- As for 'using' the application, alongside the above, there are also clear links to "register" which is the first action taken by a future user.
- When registered, a user is brought directly to their new profile page, where a message will indicate what stage of the competition is currently running and what action the user can/should take.
-
Read the competition rules and how to enter and have these be clearly explained. - PASS
- The competition rules are available for viewing on the landing page for unregistered users.
- They are also available on the compete page for viewing alongside the entry upload form.
-
View an application that is visually and creatively appealing and physically easy to look at. - PASS
- The colours and layout have all been designed so as to enhance the visual / aesthetic appeal of the application, with a specific eye to contrast and accessibility concerns as will be detailed below.
-
Browse images entered by other people to get a sense of what the application does and how it runs. - PASS
- The browse page is fully functional and allows both logged in an guest users to browse images.
-
Filter my browsing by keyword, or by selecting only images that have won awards. - PASS
- The browse page allows logged in and guest users to search for and filter photos by keywords, competition categories & awards status.
-
View the most recent winning images and see how many points they got and who they were created by. - PASS
- The "winners" page lays out the most recent award-winning images & creators and allows users to click on an image to see more details.
-
Register an account using my email and password. - PASS
- Registration works perfectly and to this original user goal, I added a username requirement to the registration.
-
Confirm my password when registering, to ensure that I don't enter a typo. - PASS
- This functionality has been implemented into the registration form.
-
Contact the application owner if I have any questions. - PASS
- The contact form functions to allow users to email the Snapathon admin directly.
click for tests
-
Login to the application. - PASS
- Login functionality works perfectly.
-
View my profile. - PASS
- Users can view their own and others' profiles.
-
Upload an avatar. - PASS
- Users can upload, change and delete their profile-pics.
-
Edit my account information - change my password, username or avatar. - PASS
- Users can update their account details including their passwords, emails, usernames and profile-pics.
-
Enter an image for competition. - PASS
- Users can upload photos to the competition as long as they remain within the entry parameters.
-
Edit the details of the image I entered for competition. - PASS
- Users can update their photo's details using the "edit photo details" button when in the photo details view page.
-
Delete the image I entered for competition. - PASS
- Users can delete any of their images using the "delete photograph" button when in the photo details view page.
-
View all the images that have been entered into this week's competition. - PASS
- Users can view all the images collated together once the entry process has ended, so when voting opens. This was specifically structured this way so as not to give anyone an unfair advantage.
- However users can also view the images in the browse section before the voting opens, but they will be mixed together with all other entries, although they can be filtered by competition theme.
-
Use my vote to vote for the image I think is the best. - PASS
- Users can vote for a single image, once voting opens.
-
See how many points I have won. - PASS
- A user's total points is displayed next to their profile photo on the profile view page.
-
See all the images I've entered into competition. - PASS
- A user can view all images they've entered into competition on their profile page under the "entries" tab.
-
View my award-winning images separately from the main collection. - PASS
- A user can view all award-winning images on their profile page under the "awards" tab.
-
View all the images I've voted for. - PASS
- A user can view all the images they have voted for on their profile page under the "votes" tab.
-
View other user profiles to see their images, who they've voted for and how many points they've won. - PASS
- A user can view other user profiles by clicking on the usernames on the winners page, by clicking on any of the thumbnails on the browse page and then clicking on the username on that photo's details page, or (should they want to) by typing "profile/<username>" in the url.
- User profiles are open to public view.
-
Browse all the images entered from all competitions. - PASS
- The browse page allows logged in and guest users to browse all entries entered into all competitions.
-
Filter my browsing by keyword, or by selecting only images that have won awards. - PASS
- The browse page allows logged in and guest users to search for and filter photos by keywords, competition categories & awards status.
-
View the most recent award-winning images and see how many points they got and who they were created by. - PASS
- The "winners" page lays out the most recent award-winning images & creators and allows users to click on an image to see more details.
click for tests
- As a user who is colourblind, I want the colours and design elements used to employ sufficient contrast so that any visual cues are easily apparent. - PASS
- The colours and contrast was taken into account during the wireframing and initial planning stages of the application.
- They were checked and rechecked with Chrome's Web Disability Simulator after every design alteration.
- The choice of yellow and dark grey was with an eye to its excellent contrast and how easily viewable it is by the greatest range of users.
- None of the application's functionality is dependent on a user being able to correctly discern between colours. Colours are used to enhance the experience for normally sighted users, such as "Delete" buttons being red, however they also have "delete" written on them, or they employ the use of icons that clearly illustrate their purpose.
- Here are some of the screenshot examples of how the application is viewed by colourblind users:
Total Colourblindness
Yellow-Blue Colourblindness
Red-Green Colourblindness
-
As a keyboard user, I want to be able to navigate the application using the keyboard. - PASS
- All functionality is fully keyboard accessible.
-
As a user using screen reader technology, I want my screen reader to describe the page elements correctly. - PASS
- Aria-labels have been added to all important elements.
- Elements have been written with an eye to the correct semantic HTML format to ensure the best possible experience for screen-reader dependent users.
click for tests
-
Create and maintain a user-friendly platform allowing photography enthusiasts to compete with each other and to inspire each other with excellent images. - PASS
- This application achieves the above.
-
Ensure that the application is as accessible as possible to include as wide a variety of users as possible. - PASS
- As above I have developed this application with an eye to strong accessibility best practices.
-
Create a competition application that is re-usable for other fields, if this one proves popular. - PASS
- The format of this application would be easy to expand on, or re-produce for other arenas/industries.
-
Eventually introduce a profit-earning aspect to the application, perhaps by monetizing awards for professional photographers .- IN DEVELOPMENT
- This has not yet been done, but remains a distinct possibility.
back to contents
When developing my search / filter method, I wanted to give users the option of filtering their searches by keywords, categories and awards. The default mongo db index $search method for $text indexes is an 'or' search, i.e. if a user types in "Mountain" and then chooses "Landscape" from the category dropdown menu, the search would return all photos entered into landscape competitions AND all images with mountain as a keyword. What I wanted is an "and" search, so that the search would return all images entered into landscape competitions with "mountain" as a keyword.
Unfortunately the mongoDB documentation was of no use here. It doesn't cover "and" searches, thankfully stack overflow had the answer. Separating the search terms with "" works as below:
{ "$search": "\"mountain\" \"landscape\"" }
When integrating pagination with my search function, it worked fine for the regular browsing page, where all images are displayed. The full number of photos returned were correctly divided up and pagination laid out, however when the search was filtered, the pagination stopped working once a user clicked to go to page 2. The initial pagination worked correctly, but then page 2 would just return all the images again, unfiltered.
I eventually changed and refactored the flask-paginate functionality into my function paginated_and_pagination_args() which worked for all pages. Thanks to Ed Bradley & Kevin from Code Institute for some initial pointers on how to go about doing this.
In order to set the values of the search form to the values searched for by the user, I needed to pass the template variables from the pages to the javascript file.
category & awards held the values of the user's search and because they represent one of multiple choices, I could not refer directly to the value in the form
field as with query.
An easy option would be to write inline JavaScript that used the template variables, however my CSP would not allow for that, without voiding the protection afforded by it.
I created two hidden elements that printed the values of category and awards to the template, and then I was able to target those elements with JavaScript in my
external script file, without altering my CSP.
When testing this function and page over the course of a week, all was working well until suddenly I was getting a 504 Gateway Time-out error message.
I used a number of print statements in the function and discovered that the issue was here:
if day_of_week in range(0,5) or day_of_week == 6 and hour_of_day < 22:
images_to_display = get_winning_images_by_week_and_year(last_week_and_year)
last_mon = week_before
while last_mon.weekday() != 0:
last_mon = last_mon - timedelta(days=1)
I had mistakenly used range(0,5) thinking that would include 5 (or weekday() == saturday), but range is not inclusive of the outer number, so it was ignoring
Saturday, and thankfully I was testing it on a Saturday, otherwise it probably would have gone unnoticed. Changing range to range(0,6) solved the issue.
As some photos are vertical and some are horizontal, the placement of the awards badge on the overlay was too far away and missing the image completely on vertical images. I needed a way to set the left: position of the award depending on whether the image was vertical or horizontal. Another related issue was the width attribute of the winning images. For landscape images I needed the width to be the full 100%, but for vertical images I needed it to be maximum 100%, as the max-height was set at 600px and by forcing the image to take up 100% of the space, much of the image would be hidden.
I used the following function to conditionally set extra classes for the horizontal images, as vertical images were the default and I could then change where the max-width was set to 100% to just be width:100%, and then I set the particular left: position for the horizontal awards as well.
function verticalOrHorizontalAwardImage(){
let photos = document.querySelectorAll('.award-photo')
photos.forEach(photo => {
if (photo.width > photo.height){
photo.classList.add('award-photo-horizontal')
let awardBadge = photo.nextElementSibling.children[0]
awardBadge.classList.add('award-horizontal')
}
})
}
The above worked, except sometimes the styles didn't seem to apply, and only on multiple page reloads would they work.
This is the photo detail page, and I wanted to dynamically set the "back to..." button to check what the source url was and then insert a link to go back to that particular page.
This was coded using request.referrer in the get_photo() route and then that was passed into the template using a "source_url" variable, which was then referenced conditionally for example:
{% if "profile" in source_url %}
<a href="{{url_for('profile', username=photo.created_by)}}"><i class="fas fa-long-arrow-alt-left"></i>Back to {{username|capitalize}}'s Profile</a>
{% elif "compete" in source_url %}
And so on... However if the user is logged in and viewing her/his own photo, they have the option of editing that photo's details. They see an "edit photo" button which brings them to the edit_photo view, they edit a form that is pre-filled with the photo's details and click save. This then brings them back to the photo detail view. With the source_url code in the template, the act of them saving changes to their image was causing the following error:
Some investigation led me to the fact that "POST" methods do not have a "source_url" insofar as the request.referrer from a POST is None, which was throwing this error. As a fix, I added another IF statement to the template, first checking IF there is a source_url, and if there is not, then it's most likely coming from the edit_photo view and does not require a "back to.." link as the user can just click on the "Edit photo details" button again.
As part of the edit profile functionality I wanted to give users the option to delete any custom profile picture they had uploaded that they no longer wanted, but without having to upload a replacement image. I wanted to include an X button on the update profile form which would allow them to just delete their current profile pic and to revert to the default.
This functionality proved a lot harder than imagined because unlike other input fields the "file" field could not automatically and easily link to the file object stored in the database, and there was no obvious way of determining how to display the delete button.
To solve this, I did a number of things:
- First in the edit_profile template, I checked whether the user had a custom profile photo uploaded and for users that did, I pre-filled the value field of the file input with the unique filename of that image.
- Then I added a delete profile pic icon with a tooltip on hover to further explain its' purpose.
- Then I devised a JavaScript function that listened for clicks on that icon and when clicked would create a hidden input field with a value to POST to my flask view. This allowed me to write logic based on the specific situation whereby a user has a profile image and wants to just delete that image.
I found that without this hidden field there were no attributes present and readable in the "file" input field that I could use to write conditional logic.
The difficulty here was in using GridFS to store the larger file type of a photograph. Mongo does not store images in their db directly, so I had to understand how the request object and gridfs work together to store files.
if 'photo' in request.files:
photo = request.files['photo']
file_id = mongo.save_file(photo.filename, photo)
The above code takes the file input with its name set to 'photo' from the request.files and sets that as a variable called photo. I then stored the result of saving that to mongo db in a variable called file_id which I was then able to add to the photo object as an attribute called file_id. Since this string is unique I could then reference it as below in order to add that specific photo's _id to the user's photos array. Hence all three collections are connected: the photos, the users and the files.
photo_to_add_to_user = mongo.db.photos.find_one({"file_id": file_id})
photo_id_to_add_to_user = photo_to_add_to_user["_id"]
The method of retrieving and displaying files that gridfs uses made this functionality more complicated, as the send_file() function that it relies on, only uses the file's "filename" to send the file. This was frustrating because it is quite possible that there could be more than one photo with the same filename. 'photo.jpg' or the like. So as I had to rely on the filenames, instead of on the objectIds as I'd hoped, I needed a way to make every filename completely unique.
I achieved this by creating a new filename, using the suffix (.png, .svg, .jpg) and then once the save_file() method had returned the file_id into my variable of the same name, I used a str() of this to create a filename for each image that is completely unique and identical to their file_id. I then updated the file in mongo to have this new filename.
filename_suffix = photo.filename[-4:]
new_filename = str(file_id) + filename_suffix
mongo.db.fs.files.update_one({"_id": file_id},
{ '$set': {"filename": new_filename}})
I wanted to implement a shuffle function for the vote page so that no one's images are given undue physical priority. For example if one image is always the first listed, and there are 50 images in the competition, which are paginated 10 per page, all images on page 1 are more than likely going to get more votes.
This proved harder than imagined to fix, first I created a shuffle function that took an array and returned a random shuffle of that array, which I then passed to the paginate function. The problem with this is that it shuffled each time a new page was loaded, so a photo might appear on page 1, but appear again on page 4. So this implementation was useless.
My next thought was to shuffle the images at source, so as they emerge from MongoDB, they would not be shuffled each time the page is loaded. But that resulted in the exact same issue, just with the shuffle happening at an earlier stage.
I realised that there is a logical inconsistency with merging a random shuffle with pagination that is difficult to overcome. If the shuffle is truly random then we are left with the initial problem that once a page is "turned" a photo can be on two pages at once, and some images may not display at all.
As a compromise, I decided to increase the number of images per page to 50 for the vote and compete pages, and although this might increase the page loading time, at least the shuffle will be consistent and all images will display, I also changed the location where the shuffle function is called.
this_weeks_entries = list(mongo.db.photos.find(
{"week_and_year": date_time.strftime("%V%G")}))
pagination, photos_paginated = paginated_and_pagination_args(
this_weeks_entries, 50, "page", "per_page")
photos_paginated_copy = photos_paginated.copy()
photos_paginated_shuffled = shuffle_array(photos_paginated_copy)
return render_template("compete.html",
this_weeks_entries=photos_paginated_shuffled,
As you can see from the above code, first I paginated the image array and then I shuffled them, this way even if there are more than 50 images and some are unlucky enough to be pushed to page 2 or further, at least within those pages the order is randomised on each page load.
This functionality will do for the first iteration of the application, but there is definitely room for improvement.
I needed a way to reference datetime in my navigation html and because the navigation html code was written in the base.html template, there was no route leading to it that I could use to include the datetime variables.
I discovered @app.context_processor functions which run before the templates are rendered and allow you to inject things into the entire application. I used a context_processor for datetime.
I got the email working after collating many online tutorials, but the "sender" information that I was extracting from the form was not translating over to gmail where the emails could be read. So it looked like all the emails were being sent from the Snapathon gmail account, as below:
I realised that this is expected behaviour, because it is the connected gmail account sending emails to itself. To pass on the sender information, I added the form sender into the message that gets delivered to the app's gmail, as below:
if request.method == "POST":
with app.app_context():
msg = Message(subject="New Email From Contact Form")
msg.sender=request.form.get("email_of_sender")
msg.recipients=[os.environ.get('MAIL_USERNAME')]
message = request.form.get("message")
--> msg.body=f"Email From: {msg.sender} \nMessage: {message}"
mail.send(msg)
flash("Email Sent!")
return redirect(url_for('home'))
The email functionality was working fine in the local port, but when deployed to Heroku I was getting the following error:
I had not input my new mail configuration variables in the Heroku config vars input area. Once I did, it connected perfectly.
When the awards function was run, it was over-awarding certain users. During testing I ran a number of simulations and found that 4 users in particular were being awarded an illogical number of points. Everyone else was being awarded the correct number of user_points, and the awards were working correctly as far as the photos were concerned.
I ran through the function line by line and using print() statements on every logical segment, I discovered that the users in question were being added twice to the valid_users array. Eventually I realised that in testing the application I had uploaded more than the one allotted photo for each of those users and since the function is based on an assumption of 1 entry per user per week, that fact was breaking the code. I deleted the offending extra images and it worked well again, but there is definitely room to refactor that code if I decide that the application could host more than a single competition per week, or if users are allowed to upload more than one image per competition.
During testing for edge-cases, I found 3 different but related scenarios that were breaking the application:
- If no one voted for any of the entries during the week and the awards() function ran on schedule on Sunday night, it was throwing an error.
- If one photo got all the votes - error.
- If one photo got say 6 votes, one photo got 4 photos and no other photos got votes, the application was awarding 1st & 2nd place logically, but then awarding 3rd place to every other entry.
When refactoring the code to account for these edge-cases, I firstly included ternary operators in my definition of all the vote_count variables. Checking to see if the arrays that they rely on had any data in them, i.e. were not 0. If they were 0, I made the count equal null. Then in defining each subsequent array, I first checked to make sure the count they relied on was not null/None.
first_place_vote_count = max(array_of_scores) if array_of_scores else None
if first_place_vote_count:
second_place_vote_array = [n for n in array_of_scores if n != first_place_vote_count]
second_place_vote_count = max(second_place_vote_array) if second_place_vote_array else None
if second_place_vote_count:
third_place_vote_array = [n for n in second_place_vote_array if n != second_place_vote_count]
third_place_vote_count = max(third_place_vote_array) if third_place_vote_array else None
return first_place_vote_count, second_place_vote_count, third_place_vote_count
In the next part of the awards logic, I then added an extra check and first_place_votes_needed > 0: to each level to ensure that the votes needed
to receive an award were greater than 0, and only if they were does the code assign awards to those images, and thus down the line, points to those users.
for entry in photo_arr:
if entry["photo_votes"] == first_place_votes_needed and first_place_votes_needed > 0:
database_var.db.photos.update_one(
{"filename": entry["filename"]},
{'$set': {"awards": 1}})
user = database_var.db.users.find_one(
{"username": entry["created_by"]})
if user not in first_place_users:
first_place_users.append(user)
413 Errors (request entity too large / Payload too large) were not passing to the error.html template to render correctly. In development I was getting a message saying the port was unresponsive.
Using print() I was able to see that the error view was working correctly right up until the rendering of the template. If I switched from rendering a template to just returning the error message like this:
@app.errorhandler(413)
def payload_too_large(e):
error = 413
error_msg = "Sorry, but the file you're trying to upload is too large."
return error_msg, 413
It worked fine, but I wanted my nicely styled error page to load, as with all other errors. Especially since this particular error would likely be thrown a lot as users try to upload large files.
After some research I found the following note in the Flask documentation:
Connection Reset Issue
When using the local development server, you may get a connection reset error instead of a 413 response.
You will get the correct status response when running the app with a production WSGI server.
I checked it on the deployed version and it still wasn't working.
Eventually after much stack overflowing, I made the executive decision to change the way the error was thrown. The preferred method is to set a configuration
variable like so: app.config['MAX_CONTENT_LENGTH'] = 750 * 750 and then application will automatically throw a 413 error when that size is surpassed by any
uploaded file.
However as I could not get this working at a level whereby the user experience was not negatively impacted, I created my own function to check the file size of images before they are saved to the database.
def check_file_size(file, size_limit_in_bytes):
file.seek(0, os.SEEK_END)
file_length = file.tell()
file.seek(0,0)
if file_length > size_limit_in_bytes:
return abort(413)
else:
return True
This worked a treat, so I was happy to use it instead of the Flask functionality, but with an eye to keeping the issue in mind for future projects. The only downside to using this method that I can see is that you must be cautious to include the function anywhere a file can be uploaded, whereas the config variable method would ensure the functionality is automatically incorporated on an application-wide basis.
However, as all my uploads are served by one save_photo() function, and that calls the check_file_size function, I can be confident it applies site-wide.
I installed flask-talisman to protect against a variety of common security threats and when I reloaded my application, it had changed it somewhat:
I gleaned that Talisman was not allowing the Materialize framework to do its job and it transpired that it was blocking a number of domains from sending data to the site, which of course is what it does. To allow in the sources of: Google Fonts, Materialize and jQuery, as well as my own JavaScript files, I had to explicity tell Talisman that those sources were ok. I did so as below with a little help from Stack Overflow (attributed in README.md)
csp = {
'default-src': [
'\'self\'',
'cdnjs.cloudflare.com',
'fonts.googleapis.com'
],
'style-src': [
"\'self\'",
'cdnjs.cloudflare.com',
'https://fonts.googleapis.com'
],
'font-src': [
"\'self\'",
"https://fonts.gstatic.com"
],
'img-src': '*',
'script-src': [
'cdnjs.cloudflare.com',
'code.jquery.com',
'\'self\'',
]
}
talisman = Talisman(app, content_security_policy=csp)
This code adds an extra layer of security as it allows in images from all sources, within the parameters of the security measures I have already set up for images, but it does not allow any other media files.
The application was not running the Scheduled processes on time. I was only able to trigger the scheduled processes within my development environment within GitPod.
There were various issues I had overlooked:
-
I needed to add a second dyno to my Procfile -->
clock: python jobs.py -
I needed to keep the application "awake" so that it would actually run the scheduled functions at the appointed times. For this I used Kaffeine a nifty little app that pings Heroku apps every 30 minutes to keep them awake.
-
I needed to change the kind of Scheduler I was using from "BackgroundScheduler" to "BlockingScheduler", this has something to do with how the schedulers exit together as per this article on SO.
-
I needed to scale the 'clock' process so there aren't multiple dynos doing the same work. This was done in the Heroku CLI using the command
heroku ps:scale clock=1 --app snapathon-comp
After implementing these changes the Scheduler worked to run the functions at the appointed times.
An issue with the functionality of the application arose when I realised that during the voting period ( Friday at midnight until Sunday at 22:00 ) users could see the points on the images by clicking into the photo details, or by looking at the profile "Votes" section of different users. Admittedly it would take some amount of independent focused research on behalf of the contestants, however if they did this, it would make it easy for a user to score points by voting for the image doing the best just before voting ends.
I looked at the locations in the application that display points and I changed the code to make the points display only occur for photos whose week_and_year field was not equal to the current week_and_year. For the photo details page this line was added to the template itself:
{% if datetime.strftime("%V%G") != photo.week_and_year %}
<h3 class="photo-points col s10 offset-s1 center-align">{{photo.photo_votes}} points</h3>
{% endif %}
For the user profile page, I confined the logic to the view rather than the template:
if photos_voted_for_array != []:
for img in photos_voted_for_array:
photo_obj = list(mongo.db.photos.find({"_id": img}))
for photo in photo_obj:
if photo["week_and_year"] != datetime.now().strftime("%V%G"):
photos_voted_for_objs.append(photo)
Where photos_voted_for_objs was the array passed to the template.
The placement of awards badges on the winner's page is defined by two classes called 'award-horizontal' and 'award-photo-horizontal'. When applied to the photo and badges of horizontal images they change the position of how the badge sits, so it looks well.
The following code was only running on page refresh and not on the initial page load.
document.addEventListener('DOMContentLoaded', function() {
function verticalOrHorizontalAwardImage(){
let photos = document.querySelectorAll('.award-photo');
photos.forEach(photo => {
if (photo.width > photo.height){
photo.classList.add('award-photo-horizontal');
let awardBadge = photo.nextElementSibling.children[0];
awardBadge.classList.add('award-horizontal');
}
});
}
verticalOrHorizontalAwardImage();
});
The awards badges were displaying too close to the centre of the image as follows:
My mentor Femi suggested I use load instead of DOMContentLoaded and I found that this change coupled with adding the
EventListener to window instead of document returned the desired functionality and the awards badges loaded correctly as below:
I am using Kaffeine to keep my application awake, so as to ensure the automated functions fire when they should. An unforeseen downside to this is that I ran out of free Dyno hours and the awards function did not fire when it should have. Dyno hours are reset each month and I will receive an extra 450 for registering a card with Heroku, however it may occur again and if the awards don't fire on a Sunday evening, this is likely why.
There is no real fix as this is not an error in the application itself. Were it running 'in production' as a working application, the extra dyno hours would be paid for fully and this error would be a non-issue.
back to contents
click for tests
- /home - PASS
- /winners - PASS
- /browse - PASS
- /profile/username - PASS
- /photos/photo_filename.jpg = PASS
- /home#contact-form - PASS
- /login - PASS
- /register - PASS
- /home - PASS
- /winners - PASS
- /browse - PASS
- /profile/username - PASS
- /edit-profile/username - PASS
- /photos/photo_filename.jpg - PASS
- /edit-photo/photo_filename.jpg - PASS
- Competition Pages
- /compete?username=username ('Compete' in navbar Mon-Fri) - PASS
- /compete?username=username ('Vote' in navbar Sat-Sun until 22:00) - PASS
- /compete?username=username ('Awards' in navbar Sun from 22:00-00:00) - PASS
- /home#contact-form - PASS
- /admin - PASS
- /home - PASS
- /winners - PASS
- /browse - PASS
- /profile/username - PASS
- /edit-profile/username - PASS
- /admin-search - PASS
- /admin-delete-user-account/username - PASS
- /photos/photo_filename.jpg - PASS
- Competition Pages
- /compete?username=username ('Compete' in navbar Mon-Fri) - PASS
- /compete?username=username ('Vote' in navbar Sat-Sun until 22:00) - PASS
- /compete?username=username ('Awards' in navbar Sun from 22:00-00:00) - PASS
- /home#contact-form - PASS
click for tests
The following url requests by the following categories of users should return a 302 redirect status code:
- /edit-profile/username - PASS
- /delete-account/username - PASS
- /edit-photo/photo_filename.jpg - PASS
- Competition Pages
- /compete?username=username ('Compete' in navbar Mon-Fri) - PASS
- /compete?username=username ('Vote' in navbar Sat-Sun until 22:00) - PASS
- /compete?username=username ('Awards' in navbar Sun from 22:00-00:00) - PASS
- /logout - PASS
- /edit-profile/username (When the username to edit is not that of the logged in user) - PASS
- /edit-photo/photo_filename.jpg (When the photo's creator is not that of the logged in user) - PASS
- /edit-photo/photo_filename.jpg - PASS
click for tests
The following url requests by the following categories of users should return a 403 Forbidden status code:
- /admin - PASS
- /admin-search - PASS
- /admin-delete-user-account/username - PASS
- /delete-account/username (When the username to delete is not that of the logged in user) - PASS
- /admin - PASS
- /admin-search - PASS
- /admin-delete-user-account/username - PASS
click for tests
Any time any user types in an incorrect URL - they should receive a 404 status code.
Specifically important is that urls that are almost correct, as in they have the correct prefix, but lead to a non-existent profile or photo, also return 404 status codes.
- /totally-incorrect-url - PASS
- /profile/incorrectusername - PASS
- /photos/incorrectphotofilename.jpg - PASS
- /totally-incorrect-url - PASS
- /profile/incorrectusername - PASS
- /photos/incorrectphotofilename.jpg - PASS
- /totally-incorrect-url - PASS
- /profile/incorrectusername - PASS
- /photos/incorrectphotofilename.jpg - PASS
- /admin-delete-user-account/incorrect-username - PASS
click for tests
If a user attempts to upload any file (image) with a filesize larger than 560KBs, the operation should be terminated and a 413 error: "Payload Too Large" should be returned.
This applies to both competition entries & profile photo uploads. The following tests were done:
- A profile photo upload attempt during user registration of a photo sized 579KBs.
- A profile photo update attempt via "Edit Profile" of a photo sized 640KBs.
- A competition entry upload attempt of a photo sized 628KBs.
These were all rejected and the correct status code & error message was returned. PASS
click for tests
A 415 "Unsupported Media Type" status code can arise in two ways:
-
If a user attempts to upload any file with an extension that is not one of the approved extensions: ['.jpg', '.png', '.gif', '.svg', '.jpeg']
-
If a user attempts to upload a file that is not an image file, even if it has the correct (fake) extension.
This applies to both competition entries & profile photo uploads. The following tests were done:
- A profile photo update attempt via the "Update Profile" form of a photo with a .exe extension.
- A competition entry upload attempt of a photo with a .pdf extension.
- A competition entry upload attempt of a pages document with a .jpg extension.
- A profile photo upload attempt via the user registration form of a photo with a .pages extension.
These were all rejected and the correct status code & error message was returned. PASS
back to contents
PASS
Testing process:
- Clicked through each navbar item to ensure they directed the user to the correct page. -- PASS
- Clicked through each sidebar (mobile navigation) item to ensure they directed the user to the correct page. -- PASS
- Checked every navigation link on the site to ensure they linked to the correct page. -- PASS
- Used W3 Link Checker to ensure there were no broken links on the page. (see details below in Validations section of this doc.) -- PASS
PASS
Testing Proces:
- Created a bunch of dummy users and then periodically logged into their accounts when testing other aspects of the code. -- PASS
- Created an admin user and logged into that account as well. -- PASS
- Used Chrome Dev Tools to ensure a new session was created. -- PASS
PASS
Testing process:
- Checked every link on the site to ensure they linked to the correct page. -- PASS
- Used W3 Link Checker to ensure there were no broken links on the page. -- PASS
PASS
Testing process:
-
Ensure that any buttons without readable text have descriptive aria-labels. -- PASS
-
Check that all form submit buttons, successfully POST the data they are meant to, by manually confirming the data's presence in the database. -- PASS
-
Ensure that buttons are used specifically to submit data and are not a substitute for links. -- PASS
-
Check that the "Vote" buttons sucessfully add 1 vote to the photograph in question. This was checked by manually verifying the votes in the database, as well as via the longer "vote testing" processes outlined below. -- PASS
-
Check that once a user has voted, that the "Vote" buttons disappear from the page. -- PASS
PASS
Testing Process:
- Submitted all forms and then immediately manually checked both the Mongo database and the application response to ensure they submitted succesffully. -- PASS
- Submitted each form with various incorrect or forbidden inputs to ensure that the form was not submitted, and that the appropriate error message was displayed to the user. -- PASS
PASS
As Materialize comes with its own very useful set of form input validations, I have used those form validations and validation messages for this application, but I have supplemented them, specifically with the custom file upload validations.
click for validations
- Email and Message are both required fields. -- PASS
- Email must match the regex pattern:
^([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,5})$-- PASS
-
Email input must be between 4 - 65 characters in length -- PASS
-
Message input must be between 5 - 1000 characters in length -- PASS
- Materialize data-length attribute used here to display maximum message length to users.
- Email is a required field. -- PASS
- Email must match the regex pattern:
^([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,5})$-- PASS
- Password is a required field. -- PASS
- Password must be between 6 - 25 characters in length. -- PASS
- Username must be present - it's a required field, cannot be left empty. -- PASS
- Username must be between 5 - 25 characters in length. -- PASS
- Email must be present - it's a required field. -- PASS
- Email must match the regex pattern:
^([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,5})$-- PASS
-
If a file is uploaded and its extension is not one of: ['.jpg', '.png', '.gif', '.svg', '.jpeg', '.heic'] then the POST should be cancelled and an error message displayed to the user. -- PASS
-
If a file is uploaded and it is above the maximum size of 750 X 750 bytes (562KB) the POST should fail and an error page displayed to the user explaining why. -- PASS
-
Password must be set. -- PASS
- Password must be confirmed. -- PASS
- Both Password & Password Confirmation must be between 6-25 characters in length -- PASS
- Passwords must match. -- PASS
- Username must be present - it's a required field, cannot be updated to blank. -- PASS
- Username must be between 5 - 25 characters in length. -- PASS
- Email is a required field. -- PASS
- Email must match the regex pattern:
^([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,5})$-- PASS
- If a file is uploaded and its extension is not one of: ['.jpg', '.png', '.gif', '.svg', '.jpeg', '.heic'] then the POST should be cancelled and an error message displayed to the user. -- PASS
- If a file is uploaded and it is above the maximum size of 750 X 750 bytes (562KB) the POST should fail and an error page displayed to the user explaining why. -- PASS
- Current password, new password & new password confirmation fields must all be between 6 - 25 characters in length. -- PASS
- If a user is trying to change their password they must enter a current password. -- PASS
- If a user is trying to change their password the current password they enter must be correct. -- PASS
- The new password must match the new password confirmation. -- PASS
- A title for the entry, the photograph itself & the camera used are all required for entry into the competition. They must not be left blank. -- PASS
-
If a file is uploaded and its extension is not one of: ['.jpg', '.png', '.gif', '.svg', '.jpeg', '.heic'] then the POST should be cancelled and an error message displayed to the user. -- PASS
-
If a file is uploaded and it is above the maximum size of 750 X 750 bytes (562KB) the POST should fail and an error page displayed to the user explaining why. -- PASS
-
There are maximum character length validations set on the following inputs: -- PASS
- Title: 45 chars -- PASS
- Story: 600 chars -- PASS
- Camera: 35 chars -- PASS
- Lens: 20 chars -- PASS
- Aperture: 20 chars -- PASS
- Shutter Speed: 20 chars -- PASS
- ISO: 20 chars -- PASS
In addition to the application not allowing the user to enter more than the maxlength validation allows for, I have also incorporated the Materialize character counter to further communicate these max-lengths to the user.
- Users must tick that they have "read and agree to the terms and conditions" before they are allowed enter the competition. -- PASS
- The title and the camera fields must remain filled. -- PASS
-
There are maximum character length validations set on the following inputs: -- PASS
- Title: 45 chars -- PASS
- Story: 600 chars -- PASS
- Camera: 35 chars -- PASS
- Lens: 20 chars -- PASS
- Aperture: 20 chars -- PASS
- Shutter Speed: 20 chars -- PASS
- ISO: 20 chars -- PASS
In addition to the application not allowing the user to enter more than the maxlength validation allows for, I have also incorporated the Materialize character counter to further communicate these max-lengths to the user.
- Users must tick that they have "read and agree to the terms and conditions" before they are allowed update their photo details. -- PASS
- Both Password & Password Confirmation must be between 6-25 characters in length -- PASS
- Current password must be entered. -- PASS
- Password must be confirmed. -- PASS
- Passwords must match. -- PASS
PASS
Testing Process:
- Ensured that the pagination links were visible on the browse page. -- PASS
- Ensured that the pagination links were visible on the vote page. -- PASS
- Ensured that clicking from pagination link 1 to 2 to 3 successfully works to display different images. -- PASS
- Checked that the pagination message correctly reflected the photo count and the page the user is currently viewing. -- PASS
PASS
Testing Process:
- Used the contact form to send the connected SNAPATHON email account a dummy email from a dummy guest user. -- PASS
- Verified its successful receipt by logging into the email account and viewing the email, as below: -- PASS
PASS
Testing Process:
- Clicked logout and then manually ensured that the user did not have access to any of the logged in functionality. -- PASS
- Visually checked that the user received the correct successful logout flash message. -- PASS
- Navigated to the user's profile and ensured that they could not access restricted functionality: the "edit profile" function for example. -- PASS
back to contents
All the various CRUD actions were thoroughly manually tested by the creation, viewing, editing and destruction of multiple dummy users and competition entries.
PASS
Testing process:
- Click on one of the various "Register" links, and verify that the register page loads.
- Fill out username, email address, profile pic & both password fields correctly.
- Click the "Register" button.
- Confirm successful redirection to the new user's profile page, with the username & profile photo displaying correctly.
- Further confirmed by manually checking the Mongo database to confirm user creation.
PASS
Testing process:
- Click on a link to the Compete page.
- Fill out the entry form including: title, uploading the entry, story, camera, lens, aperture, shutter speed, iso and checking the disclaimer.
- Click on the "Compete" button.
- Verify that a successful submission screem appears displaying the uploaded entry.
- Verify that the uploaded image is visible on the user's profile page.
- Further confirmed by checking that a new entry was created in the Mongo database.
PASS
Testing process:
- Navigate to the "recent winners" page.
- Click on a winner's username to view their profile page.
- Navigate to the "Browse Images" page.
- Click on an image.
- On the image details page, click on the username of the image's creator.
- View the profile page.
PASS
Testing process:
- Navigate to the "Browse" page.
- Click on any image to view its image details page.
- Click back to the browse images page, using the "back to browse" link.
- Click the pagination links to verify they work.
- Filter a search by searching for award-winning images only.
- Reset the search.
- Filter a search by searching for landscape images only.
- Reset the search.
- Filter a search by searching for images associated with the username: "karina".
- Reset the search.
- Filter a search by searching for archictectural images that have won awards and have the keyword "sky" attached to them.
PASS
Testing process:
- Navigate to the "Browse" page.
- Click on any image to view its image details page.
- Navigate to that photo's user.
- Click on another of their photos to be brought to that photo details page.
- Navigate to the Winner's Page.
- Click on one of the winning photos to be brought to that photo details page.
PASS
Testing process:
- Navigate to the "Winners" page.
- Scroll down and click on an image.
- Navigate back to the winner's page.
- Click on one of the winning photo's user's name.
- Verify the photo is displayed in the "Awards" tab on their user profile page.
PASS
Testing process:
- Login as admin
- Should be automatically redirected to the admin user control panel.
- Click on the pencil button to see the specific update user screen for admin users.
PASS
Testing process:
- In the user control panel type a username and click search
- View the results.
- Reset the search fields.
PASS
Testing process:
- As a logged in user, click into any of their images to view the photo details page.
- Click "Edit photo details"
- Edit the image details.
- Click "Update Photo Details"
- Verify that the correct changes have been made by viewing the photo details page.
PASS
Testing process:
- As a logged in user viewing their profile page, click "edit profile".
- Edit the profile in some way.
- Click "Save Changes"
- Verify that the correct changes have been saved by viewing the profile page.
PASS
Testing process:
- As a logged in admin user on the admin user control page, click "edit profile".
- Edit the profile in some way.
- Click "Save Changes"
- Verify that the correct changes have been saved by viewing the profile page.
PASS
Voting could arguably be a "create" or an "update" process, but as this application's voting functionality essentially updates the photo document in mongo, I will categorise it as an update.
Testing process:
- Click on a link to the Vote page.
- Click on a "vote" button to vote for a particular image.
- Verify that "Thank you for voting" & "you have voted" messages are displayed.
- Verify that the vote buttons have disappeared.
- Verify that the photo voted for has a yellow "you voted for this image" overlay.
- Further confirmed by checking that the photo voted for has both been awarded an extra vote and that the user who voted for that image, now has that photo object id added to her photos_voted_for array in the Mongo database.
PASS
Testing process:
- As a logged in user on one of their photo details page, click "delete photograph".
- Confirm the deletion by clicking "yes, delete it".
- Verify that the photograph has been deleted.
- Double check the deletion by verifying that the photo is no longer in the "photos" collection, the "files" collection and the "chunks" collection in the Mongo database.
- If the user deletes a photo uploaded into the current week's competition and the day of the week is still between Mon-Fri, then the user should be able to upload a new image. This was tested by doing the latter.
PASS
Testing process:
- As a logged in user on their profile page, click "edit profile".
- Click "delete account".
- Enter the user password twice.
- Account is deleted and user session ended.
- Verify the deletions in the database: the user, their profile photo, their entries - all successfully deleted from users, photos, files & chunks collections.
PASS
Testing process:
- As a logged in admin user on the admin user control page, click delete button next to the profile to be deleted.
- Click "delete account".
- Enter the admin password twice.
- Confirm the account deletion.
- Check that the account has been deleted from admin user control.
- Verify the deletions in the database: the user, their profile photo, their entries - all successfully deleted from users, photos, files & chunks collections.
back to contents
As many of the pages and application content changes depending on the day of the week and hour of the day, my manual testing needed to include a temporal element to ensure that the pages were behaving as planned and on schedule.
Testing Process:
Open up the application on Saturday and ensure that all of the following has happened:
- The "compete" page becomes a "vote" page. -- PASS
- The vote page displays all of the past week's entries with clickable "Vote" buttons underneath them. -- PASS
- The navbar link for this page changes from "compete" to "vote" -- PASS
Testing Process:
Open up the application on Sunday after 22:00PM and ensure that all of the following has happened:
- The awards and results are automatically calculated using APScheduler. -- PASS
- The winners page is updated to display this week's awards and results. -- PASS
- The "vote" page turns into an interim "holding" page directing users over to the winners page to view the results. -- PASS
- The navbar link changes from "vote" to "awards". -- PASS
-
The messages on a user's profile page are specific to their individual interaction with the application.
-
If it is between Monday & Friday and the user has not entered the competition, they should see the following message:
"You still have to enter an image into this week's competition. You have 2 days, 3 hours and 34 minutes left to enter."
PASS
-
If it is between Monday & Friday and the user has entered the competition, they should see the following message:
"Thank you for entering the competition. Voting opens in 4 days, 4 hours and 12 minutes."
PASS
-
If it is between Saturday & Sunday before 22:00 and the user has entered the competition and has voted, they should see the following message:
"Thank you for voting! Voting ends at 22:00 this Sunday in 0 days, 4 hours and 41 minutes, when awards, points & winners will be announced."
PASS
-
If it is between Saturday & Sunday before 22:00 and the user has entered the competition but has not voted, they should see the following message:
"You still have to vote for your favourite image. You have 0 days, 4 hours and 40 minutes left to vote. If you don't vote, your entry's points will be reduced to 0. Go to VOTE and cast your vote!"
PASS
-
If it is between Saturday & Sunday before 22:00 and the user has not entered the competition, they should see the following message:
"You did not enter this week's competition and therefore cannot vote. Voting ends at 22:00 this Sunday in 0 days, 2 hours and 3 minutes, when awards, points & winners will be announced."
PASS
-
If it is Sunday after 22:00 all users will see the following message:
"The winners have been announced! Head to WINNERS to see the results!"
PASS
back to contents
The awards() function runs automatically on a Sunday evening at 22:00 - and for the first tests I decided to change those settings and run it manually.
Firstly I created a selection of dummy users and for each of them I entered 1 image into a dummy weekly competition. I then made each user vote for various images and recorded who voted for which image. I created two spreadsheets: one recording what the user actions were and expected results, the other recording the photograph votes received and expected awards. Basically a manual version of what automated tests would achieve.
| # | Photo Title | Votes Received | Photo Created By | Users Who Voted For Photo | Expected Awards | Actual Awards |
|---|---|---|---|---|---|---|
| . | Test 1 | |||||
| 1. | "Sunset of Fire" | 7 | Eoghan | Anne1, Cathy, Frederick, Loretta, Monica, Orlaith, Derrick | 1st place | 1st place |
| 2. | "Best Beach Ever" | 3 | Frederick | Eoghan, Georgina, Horatio | 2nd place | 2nd place |
| 3. | "Lightening Attack" | 3 | Ignacio | Barbara, Stephanie, Quentin | 2nd place | 2nd place |
| 4. | "Peace & Quiet" | 2 | Derrick | Ignacio, Patricia | 3rd place | 3rd place |
| . | Test 2 | |||||
| 1. | "Sunset of Fire" | 7 | Eoghan | Anne1, Cathy, Frederick, Loretta, Monica, Orlaith, Derrick | 1st place | 1st place |
| 2. | "Best Beach Ever" | 3 | Frederick | Eoghan, Georgina, Horatio | 2nd place | 2nd place |
| 3. | "Lightening Attack" | 3 | Ignacio | Barbara, Stephanie, Quentin | 2nd place | 2nd place |
| 4. | "Peace & Quiet" | 2 | Derrick | Ignacio, Patricia | 3rd place | 3rd place |
As you can see the awards and points logic functioned perfectly in both tests, but as below illustrates this manually testing strategy caught an inconsistency with the user_points. Once I solved it, test 2 ran correctly.
| # | User | User Photo | Photo User Voted For | Expected User Points From Awards | Expected User Points From Voting | Expected User Points Total | Actual User Points Total |
|---|---|---|---|---|---|---|---|
| . | Test 1 | ||||||
| 1. | Derrick | "Peace & Quiet" | "Sunset of Fire" | 3 | 3 | 6 | 6 |
| 2. | Eoghan | "Sunset of Fire" | "Best Beach Ever" | 7 | 2 | 9 | 11 |
| 3. | Frederick | "Best Beach Ever" | "Sunset of Fire" | 5 | 3 | 8 | 11 |
| 4. | Ignacio | "Lightening Attack" | "Peace & Quiet" | 5 | 1 | 6 | 6 |
| . | Test 2 | ||||||
| 1. | Derrick | "Peace & Quiet" | "Sunset of Fire" | 3 | 3 | 6 | 6 |
| 2. | Eoghan | "Sunset of Fire" | "Best Beach Ever" | 7 | 2 | 9 | 9 |
| 3. | Frederick | "Best Beach Ever" | "Sunset of Fire" | 5 | 3 | 8 | 8 |
| 4. | Ignacio | "Lightening Attack" | "Peace & Quiet" | 5 | 1 | 6 | 6 |
I created two development functions clear_user_points() & clear_all_awards(), to quickly and easily clear the slate and re-test the awards() function as many times as needed.
def clear_user_points():
all_users = list(mongo.db.users.find())
for user in all_users:
mongo.db.users.update_one({"username": user["username"]}, {'$set': {"user_points": 0}})
print("All user points zeroed")
def clear_all_awards():
all_photos = list(mongo.db.photos.find())
for photo in all_photos:
mongo.db.photos.update_one({"filename": photo["filename"]}, {'$set': {"awards": None}})
print("No photo has any awards now.")
This strategy helped me catch one issue that arose not because of the code logic, but because I had allowed 4 users to upload more than one image.
The rules of the competition state that if you enter the competition, you must vote for an image other than you own before 22:00PM on Sunday. Users who enter and who do not vote, will have their entry's points reduced to 0. This happens automatically as votes are counted. To test this, I created a dummy user called "Franny" who entered an image of some leaves, and whose image got 100 votes between Friday night and Sunday at 22:00. Franny did not vote for any image. I then ran the awards() function and made sure that her image's points were reduced to 0.
.png)
.png)
.png)
.png)
As you can see the awards() function correctly reduced Franny's votes_to_use from 1 to 0,
as well as reducing her photo "leaves"'s photo_votes from 100 to 0.
Leaves received more votes than any other photo in that competition, but did not win any awards,
evidenced by its awards field remaining null
To ensure that everything including the AP Scheduler automaton worked well, I ran another test, but for this one I did not interfere with the timing of the logic.
-
Again, I created (and re-used) 32 dummy users and images and had them all enter a weekly competition with the theme of "Wildlife".
-
I ensured that all the awards() logic was as it would be in the final deployment.
-
On Saturday I had 30 of the dummy users vote for their favourite images.
-
1 dummy user (harriet) "forgot" to vote and she received 7 votes on her image which would have been enough to give her the 1st place award. The expectation is that her photo's points are reduced to 0.
-
Harriet's image will be disqualified, its total points reduced to 0 and it will not receive an award.
-
Eoghan1's image "Zebby" will come first with 6 points.
-
Yasmine's image "Macaw" will come second with 4 points
-
Patricia's image "Turtle" will come third with 3 points.
-
Other expected user points are outlined below in the chart used to keep track of scoring.
As you can see in the chart below, the function worked perfectly and all the predicted test results were correct. Harriet's entry was invalidated and the scoring continued unfazed to assign 1st, 2nd & 3rd and to calculate all the user points correctly.
The results were checked firstly by going to the "Winners" page where the correct winning images & users were displayed.
Then the users were individually verified in the Mongo database to ensure that everyone received the correct awards.
As everything was spot on, no further tests were needed for this function.
| # | User | User Photo | Photo User Voted For | Expected User Points From Awards | Expected User Points From Voting | Expected User Points Total | Actual User Points Total | Expected Awards | Actual Awards |
|---|---|---|---|---|---|---|---|---|---|
| 1. | Georgina | Night Swimming | kitten | 0 | 0 | 0 | 0 | 0 | 0 |
| 2. | Annie | Mr. Frederickson | zebby | 0 | 3 | 3 | 3 | 0 | 0 |
| 3. | Anne1 | Humming Bird | snow on cat | 0 | 0 | 0 | 0 | 0 | 0 |
| 4. | Cathy | Robin in tree | turtle | 0 | 1 | 1 | 1 | 0 | 0 |
| 5. | Eoghan1 | Zebby | dangerously pretty | 7 | 0 | 7 | 7 | 1 | 1 |
| 6. | Horatio | Hare today | macaw | 0 | 2 | 2 | 2 | 0 | 0 |
| 7. | Ignacio | Foxy | turtle | 0 | 1 | 1 | 1 | 0 | 0 |
| 8. | Jonathan | Birds of a Feather | oil painting | 0 | 0 | 0 | 0 | 0 | 0 |
| 9. | Loretta | oil painting | zebby | 0 | 3 | 3 | 3 | 0 | 0 |
| 10. | Monica | grass in the darkness still green | kitten | 0 | 0 | 0 | 0 | 0 | 0 |
| 11. | Nicola | huffin' and puffin' | kitten | 0 | 0 | 0 | 0 | 0 | 0 |
| 12. | Orlaith | red | macaw | 0 | 2 | 2 | 2 | 0 | 0 |
| 13. | Patricia | turtle | zebby | 3 | 3 | 6 | 6 | 3 | 3 |
| 14. | Quentin | singing | kitten | 0 | 0 | 0 | 0 | 0 | 0 |
| 15. | Roberta | stripes | mr.frederickson | 0 | 0 | 0 | 0 | 0 | 0 |
| 16. | Franny | gliding | kitten | 0 | 0 | 0 | 0 | 0 | 0 |
| 17. | Stephanie | pride of parnell | oil painting | 0 | 0 | 0 | 0 | 0 | 0 |
| 18. | Tristan | dangerously pretty | singing | 0 | 0 | 0 | 0 | 0 | 0 |
| 19. | Ursula | elephant | zebby | 0 | 3 | 3 | 3 | 0 | 0 |
| 20. | Victoria | black tipped reef shark | huffin & puffin | 0 | 0 | 0 | 0 | 0 | 0 |
| 21. | Karina | jellyfish | macaw | 0 | 2 | 2 | 2 | 0 | 0 |
| 22. | Xavier | palomino pony | turtle | 0 | 1 | 1 | 1 | 0 | 0 |
| 23. | Eoghan | cry of the moose | palomino pony | 0 | 0 | 0 | 0 | 0 | 0 |
| 24. | Yasmine | Macaw | kitten | 5 | 0 | 5 | 5 | 2 | 2 |
| 25. | Anthony | Fishypoo | jellyfish | 0 | 0 | 0 | 0 | 0 | 0 |
| 26. | Guillerme | Flocking Geese | zebby | 0 | 3 | 3 | 3 | 0 | 0 |
| 27. | Davina | Show Off | robin in tree | 0 | 0 | 0 | 0 | 0 | 0 |
| 28. | Henrietta | Tami the Horse | red | 0 | 0 | 0 | 0 | 0 | 0 |
| 29. | Harriet | Kitten | DIDN'T VOTE | - | - | - | - | - | - |
| 30. | Gareth | Fish Shoal | kitten | 0 | 0 | 0 | 0 | 0 | 0 |
| 31. | Bobby | Snow on Cat | macaw | 0 | 2 | 2 | 2 | 0 | 0 |
| 32. | Dominic | Cristmas Bird | zebby | 0 | 3 | 3 | 3 | 0 | 0 |
Obviously the voting process is tested alongside both above awards tests. However I also conducted more manual testing on the voting process.
- During the voting days (Sat-Sun), I used a number of dummy users that had previously (that week) entered the competition.
- I got each of them to vote for particular images and then I manually checked the database to ensure 4 things happened:
- The image voted for in each case had its' "photo_votes" field incremented by 1 each time.
- The user in question had their "votes_to_use" field decremented by 1, from 1 to 0.
- That the photo voted for did not have their "points" displayed anywhere on their photo details page.
- That the photo voted for was not displayed in the voter's "VOTE" gallery. (This is only updated after the competition ends).
This was true in every case, so in addition to the functionality outlined by the awards() tests, I could be sure that the voting was working as expected.
PASS
back to contents
CSRF is hard to test without actually attacking the site, which is outside the scope of this project. However I've listed below some of the checks I implemented so that I can reasonably assume that the site is protected.
- No important or site altering functionality is served by GET requests.
- Deleting accounts is a POST request that further requires password confirmation.
- All POST forms are embedded with CSRF tokens from WTF-Forms.
- The app is wrapped with:
csrf = CSRFProtect(app)
Another security validation incorporated is the werkzeug secure_filename util. This is applied to the uploaded photo filename before it is saved to the database, to ensure that any dodgy filenames e.g. /paths/to/os/systems/etc.jpg are sanitized before they can do any damage.
To test this I tried to upload various files with dodgy extensions.
Werkzeug does its job by transforming them with _ underscores e.g.
- malicious/paths/os/users.jpg --> malicious_paths_os_users.jpg
- ../../users/delete.png --> users_delete.png
Rendering them relatively harmless.
On top of that, the application code also works to rename the files anyway, to make them unique and to aid with the database connections.
The application only accepts file uploads with the extensions of: ['.jpg', '.png', '.gif', '.svg', '.jpeg'].
- I attempted to upload a host of other file types such as .doc, .pdf, .exe & .zip
- I verified that the upload did not work and that the POST method was terminated.
PASS
A very important check on the file uploaded, is the imghdr.what() that verifies that the filetype is an image. This extends the
Approved File Type Extension Security Measure insofar as that measure checks the visible extension, but this one goes further to check
the actual file type.
To test the functionality of this security code, I first created a .pages file and changed the extension to .jpg With the code commented out, I tried to upload this file and it uploaded successfully as you can see below:
Then I un-commented the file type validation code and after deleting the fake image, I tried again to upload it. This time, the .pages masquerading as a .jpg was duly rejected and a 415 error was correctly returned to the user, as below:
PASS
The application only accepts files that are under 750 X 750 bytes in size. This is a useful validation for two reasons: it stops overly massive images from being uploaded that would slow the application down, and it stops a decent amount of potential security threats where hackers upload malicious programmes.
To test this I tried to upload larger files and the upload forms, whether they were register, update profile or to enter an image into the competition, were rejected and the process halted.
PASS
Flask-Talisman adds a host of security features out of the box. These are described in detail in the README.md doc.
First I checked the Security Overview in Chrome Dev Tools:
Then to test the overall efficacy of Flask-Talisman I used Observatory by Mozilla - a super useful security tool that scans your site and reports on vulnerabilities. It lists all the key elements of security that Flask-Talisman integrates (as well as many others) and checks that they are functional.
When fully secured, the site scored 105/100 (an A+) and passed 10 of the 11 tests, which I deemed good enough.
Most notably the usefulness of Observatory is that it enabled me to notice where there were vulnerabilities and to improve the rating from a 65/100 score as shown below:
PASS
One of the most important aspects of security for any application using user logins is to ensure that other users cannot access private user accounts.
To test that user accounts are secure, I attempted to access the private pages of both regular users and admin users. I did so both as a guest user and logged in a regular user and as an admin user.
These have been previously covered in the status code testing section, but to reiterate from a security standpoint:
As a guest user I manually typed in the urls for:
- /admin - Access denied. -- PASS
- /edit-profile/username - Access denied. -- PASS
- /edit-photo/filename - Access denied. -- PASS
- /admin-delete-user-account/username - Access denied -- PASS
As a regular user I manually typed in the urls for:
- /admin - Access denied. -- PASS
- /edit-profile/username - Access denied. -- PASS
- /edit-photo/filename - Access denied. -- PASS
- /admin-delete-user-account/username - Access denied -- PASS
As an admin user I manually typed in the urls for:
- /edit-photo/filename - Access denied. -- PASS
The delete functions to delete accounts & photos were purposefully written as POST methods for added security, so they cannot be accessed via url.
back to contents
In addition to the accessibility user story testing outlined above, I also undertook the following manual & automated tests:
The application scored very highly on the lighthouse measure of accessibility in Chrome Dev Tools:
This is a useful tool for getting a quick overview of areas that might need some accessibility attention. It highlights areas of concern as well as good accessibility practice and shows the code it references, as below:
Another automated accessibility checker that looks at a range of identifiers. Snapathon scored well:
-
Everytime I added functionality I tested it to make sure it was fully keyboard accessible.
-
The application works perfectly for users relying solely on their keyboard. PASS
-
I used Apple's voice over utility to test the screen reader accessibility of the site.
-
Which is how I discovered a surprising issue with Chrome specifically. When a text field receives focus and Chrome offers up suggestions of what to type in that field. (Something it does a lot), this interferes with the screen reader's ability to read the label of that text input. Instead of, for example: "Filter by Keyword" the screen reader will announce "Candidate List Shown".
-
Thus many of the form labels are not read correctly by screen readers for users using Chrome.
-
Having researched it, there doesn't seem to be alot of options for developers in terms of getting around it, the onus is on Chrome to sort it out. Presumably users who use screen readers are aware of the issue and probably use a browser like FireFox where everything works perfectly.
-
The ChromeVox extension does seem to be aware of this, as the issue does not occur when a user uses it, but as an added issue, the ChromeVox seems to be programmed to ignore "Skip to Main" links.
-
The other issue I discovered with ChromeVox is that it interfered with keyboard access to the extent that I couldn't select to "Delete Account" when the extension was active.
-
In summary Chrome does not appear to be the best browser for users that rely on screen-reader technology.
-
Other than these Chrome adventures, the application is fully screen-reader accessible. PASS
back to contents
The application's responsivity and functionality was tested on all major browsers with the exception of Internet Explorer, as its usage is so low and it is due to be completely retired by 17th August 2021.
Here are the results of the browser testing using Browser Stack:
click for test results
| OS | Browser | Version | Design Check | Functionality Check |
|---|---|---|---|---|
| Windows 7, 8, 8.1 & 10 | Microsoft Edge | 89 (latest) / 90 (dev) | ✓ | ✓ |
| 90 | ✓ | ✓ | ||
| 89 | ✓ | ✓ | ||
| 86 | ✓ | ✓ | ||
| 85 | ✓ | ✓ | ||
| 84 | ✓ | ✓ | ||
| 83 | ✓ | ✓ | ||
| 82 | ✓ | ✓ | ||
| 81 | ✓ | ✓ | ||
| 80 | ✓ | ✓ | ||
| 18 | ✓ | ✓ | ||
| 17 | ✓ | ✓ | ||
| 16 | ✓ | ✓ | ||
| 15 | ✓ | x | ||
| Windows 7, 8, 8.1 & 10 & | Firefox | 84 (latest) | ✓ | ✓ |
| Mac OSX Mavericks and Newer | 86 / 87 (beta) | ✓ | ✓ | |
| 87 | ✓ | ✓ | ||
| 86 | ✓ | ✓ | ||
| 85 | ✓ | ✓ | ||
| 84 | ✓ | ✓ | ||
| 83 | ✓ | ✓ | ||
| 82 | ✓ | ✓ | ||
| 81 | ✓ | ✓ | ||
| 80 | ✓ | ✓ | ||
| 79 | ✓ | ✓ | ||
| 78 | ✓ | ✓ | ||
| ↑ | ↑ | 36 | ✓ | ✓ |
| ↓ | ↓ | 35 | X | ✓ |
| Windows 7, 8, 8.1 & 10 & | Chrome | 87 (latest) | ||
| Mac OSX Mavericks and Newer | 86 | ✓ | ✓ | |
| 85 | ✓ | ✓ | ||
| 84 | ✓ | ✓ | ||
| 83 | ✓ | ✓ | ||
| 82 | ✓ | ✓ | ||
| 81 | ✓ | ✓ | ||
| 80 | ✓ | ✓ | ||
| ↑ | ↑ | 49 | ✓ | ✓ |
| ↓ | ↓ | 48 | X | ✓ |
| Windows 7, 8, 8.1 & 10 & | Opera | 73 (latest) | ✓ | ✓ |
| Mac OSX Mavericks and Newer | 72 | ✓ | ✓ | |
| 71 | ✓ | ✓ | ||
| 68 | ✓ | ✓ | ||
| 67 | ✓ | ✓ | ||
| 66 | ✓ | ✓ | ||
| 65 | ✓ | ✓ | ||
| ↑ | ↑ | 36 | ✓ | ✓ |
| ↓ | ↓ | 35 | X | ✓ |
For Mac systems nothing older than OSX Mavericks could sucessfully run the application, and for Windows nothing older than Windows 7.
- On Windows desktops it worked perfectly on all versions of Microsoft Edge.
- On Windows Microsoft Edge v15 object-fit doesn't work so functionality and design are compromised.
- On Windows & Mac desktops it worked on all versions of Firefox from v36 upwards.
- On Windows & Mac desktops Firefox v35 and down, object-fit no longer functions.
- On Windows & Mac desktops it worked on all versions of Chrome from v49 upwards.
- On Windows & Mac desktops it worked on all versions of Opera from v36 upwards.
Below are just a selection of screenshots from the desktop browser testing:
Windows 10 - Edge - OS 15: -- FAIL
Windows 10 - Edge - OS 89: -- PASS
Mac - Firefox - OS 67: -- PASS
Mac - Chrome - OS 48 : -- FAIL
Mac - Chrome - OS 49 : -- PASS
Windows - Opera - OS 35 : -- FAIL
Windows - Opera - OS 47 : -- PASS
click for test results
The application's responsivity and functionality were tested across a wide platform of mobile devices, operating systems and mobile browsers as outlined below:
| Mobile Device | OS | Browser | Design Check | Functionality Check |
|---|---|---|---|---|
| iPhone & iPad | 14 | Safari & Chrome | ✓ | ✓ |
| 13 | Safari & Chrome | ✓ | ✓ | |
| 12 | Safari & Chrome | ✓ | ✓ | |
| 11 | Chrome | ✓ | ✓ | |
| 10 | Chrome | ✓ | ✓ | |
| ↓ | 11 | Safari | X | X |
| ↓ | 9 | Chrome | X | X |
| Samsung Galaxy S20, S9, S8, S10+, S10e, S9+, S8+, S7, S6, S5, S4, A51, A11, A10, A8, Note 20 - Note3 | 10, 9, 8, 7, 6, 5, 4.4, 4.3 | Chrome, Firefox, Samsung Internet & UC Browser | ✓ | ✓ |
| Google Pixel 5, 4, 3a, 3, 2, Pixel, PixelXL, Nexus6P, Nexus 6, 5, 9, 7 | 11, 10, 9, 8, 7.1, 7, 6, 5, 4.4, 5.1, 6 | Chrome, Firefox & UC Browser | ✓ | ✓ |
| OnePlus 8, 7T, 7, 6T | 10, 9 | Chrome, Firefox & UC Browser | ✓ | ✓ |
| Moto G7 Play, Moto X 2nd Gen, Moto G 2nd Gen | 9, 6, 5 | Chrome, Firefox & UC Browser | ✓ | ✓ |
| Xiaomi Redmi Note | 8 | Chrome, Firefox & UC Browser | ✓ | ✓ |
| Vivo Y50 | 10 | Chrome & Firefox | ✓ | ✓ |
| Oppo Reno 3 Pro | 10 | Chrome & Firefox | ✓ | ✓ |
As the above demonstrates there are issues running the application on Apple devices from versions 9 on Chrome & 11 on Safari.
Androids are far more backwards compatible, and using Browser Stack I found zero incompatibility testing the devices and OSs listed above.
Below is a selection of randomly selected screenshots of the application successfully running on a spectrum of mobile devices:
iPhone IOS9 - Chrome : -- FAIL
iPhone IOS11 - Chrome : -- PASS
iPad IOS12 - Safari : -- PASS
Android OS 4.4 - Google Nexus : -- PASS
Android OS 4.3 - Samsung Galaxy Note 3 : -- PASS
Android OS 5 - Moto G2 : -- PASS
Android OS 9 - One Plus 7 : -- PASS
Android OS 9 - Xiaomi Redmi Note 8 : -- PASS
Android OS 10 - Oppo Reno 3 : -- PASS
Android OS 11 - Samsung Galaxy s21 : -- PASS
back to contents
In addition to the extensive testing outlined in the responsivity section in this application's README.md file, I also used Chrome's Responsive Viewer for a great overview of how different pages act responsively, and where breakpoint issues arose.
These tests were part-automated insofar as the responsive viewer enabled me to test the app simultaneously across a number of devices.
Finally I also employed the standard Chrome Dev Tools Mobile emulator, but with a certain caution, as I had already discovered from previous projects that it is not 100% accurate when emulating functionality. For that, I relied more on actually using as many different devices as I could get my hands on, along with asking friends and family who use different phones and tablets to test the app for me.
back to contents
This proved quite useful. I found that Jinja templating made it easy to end up with broken tags. The validator caught these and I refactored appropriately.
Some application-wide warnings raised:
-
There was a warning that the content security policy was a bad one. But since the application scored so highly during security testing and since this issue has also been raised by other developers, it's not an issue.
-
The validator also flagged that the Flash Messages section had no header, which of course is intentional, thus is also of no concern.
Another issue I found was that URI link testing only worked for non-protected pages, so for the others I had to login and then copy and paste the html directly into the validator.
click for test results
- The aria-label is not "misused" as in the warning as it is needed to declare the checkboxes label for screen-reader dependent users.
- Passed with no issues and no invalid links or anchors.
- Two non-issues were raised as errors:
- The validator claimed that "text-decoration-thickness" doesn't exist, but as of 2019 it does exist.
- "Value Error : background 92% is not a color-stop value )" - Again the validator is incorrect and a quick Stack Overflow found multiple developers complaining about [this].(https://stackoverflow.com/questions/64754909/css-validator-error-value-error-background-100-is-not-a-color-stop-value)
- Multiple "Warnings" were raised, all of which were about vendor extensions.
So other than the above issues that can safely ignored, the CSS validated perfectly.
PASS
As always JSHint was invaluable for telling me about the thousands of semi-colons I had omitted.
Here is the first run of JSHint:
Here is the second run, semi-colons in their rightful places:
The rest of the warnings concerned the use of ES6, so were fine.
The python code all validated perfectly.
back to contents
As illustrated below, the final application scored highly in all categories of the Lighthouse Tests. The accessibility results have been covered above, but the remaining tests are here:
